{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "velvet-sussex",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "origin\tgit@github.com:ustchope/dlwpn.git (fetch)\n",
      "origin\tgit@github.com:ustchope/dlwpn.git (push)\n",
      "[master 4df8f1d] 更新ch7 #5 change 2021-07028\n",
      " 2 files changed, 1526 insertions(+), 408 deletions(-)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "To github.com:ustchope/dlwpn.git\n",
      "   6ca0969..4df8f1d  master -> master\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 6.23 s (started: 2021-07-28 21:59:08 +08:00)\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "# 增加更新\n",
    "git add .\n",
    "\n",
    "git remote -v\n",
    "\n",
    "git commit -m '更新ch7 #5 change 2021-07028'\n",
    "\n",
    "git push origin master"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "extraordinary-submission",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 6.65 s (started: 2021-07-28 20:38:15 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 自动计算cell的计算时间\n",
    "%load_ext autotime\n",
    "\n",
    "#设置使用的gpu\n",
    "import tensorflow as tf\n",
    "\n",
    "gpus = tf.config.list_physical_devices(\"GPU\")\n",
    "\n",
    "if gpus:\n",
    "   \n",
    "    gpu0 = gpus[2] #如果有多个GPU，仅使用第0个GPU\n",
    "    tf.config.experimental.set_memory_growth(gpu0, True) #设置GPU显存用量按需使用\n",
    "    # 或者也可以设置GPU显存为固定使用量(例如：4G)\n",
    "    #tf.config.experimental.set_virtual_device_configuration(gpu0,\n",
    "    #    [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=4096)]) \n",
    "    tf.config.set_visible_devices([gpu0],\"GPU\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cloudy-designer",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 32.6 ms (started: 2021-07-28 20:38:22 +08:00)\n"
     ]
    }
   ],
   "source": [
    "%config InlineBackend.figure_format='svg' #矢量图设置，让绘图更清晰"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "organic-governor",
   "metadata": {},
   "source": [
    "# 使用Keras：深入探讨"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accessory-calendar",
   "metadata": {},
   "source": [
    "**本章包含**\n",
    "* 创建 Keras 模型的不同方法：Sequential 类、Functional API 和模型子类化\n",
    "* 如何使用内置的 Keras 训练和评估循环——包括如何使用自定义指标和自定义损失\n",
    "* 使用 Keras 回调进一步自定义训练的进行方式\n",
    "* 使用 TensorBoard 监控您的训练和评估指标随着时间的推移\n",
    "* 如何从头开始编写自己的训练和评估循环"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "decent-movie",
   "metadata": {},
   "source": [
    "您开始对 Keras 有了一定的经验——熟悉 Sequential Dense 模型、层和用于训练、评估和推理的内置 API——compile()、fit()、evaluate()、 和 predict() 。 您甚至在第 3 章中学习了如何从 Layer 类继承以创建自定义层，以及如何使用 TensorFlow GradientTape 实现逐步训练循环。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "weird-establishment",
   "metadata": {},
   "source": [
    "在接下来的章节中，我们将深入研究计算机视觉、时间序列预测、自然语言处理和生成式深度学习。 这些复杂的应用程序将需要更多\n",
    "比顺序架构和默认的 fit() 循环。 所以让我们先把你变成 Keras 专家！ 在本章中，您将全面了解使用 Keras API 的关键方法：处理接下来将遇到的高级深度学习用例所需的一切。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fewer-consultancy",
   "metadata": {},
   "source": [
    "## 工作流程"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "potential-there",
   "metadata": {},
   "source": [
    "Keras API 的设计遵循逐步公开复杂性的原则：易于上手，但可以处理高复杂性的用例，只需要在每一步进行增量学习。 简单的用例应该是简单易懂的，任意高级的工作流程应该是：无论你想做的事情多么小众和多么复杂，都应该有一条清晰的路径。 一条建立在您从更简单的工作流程中学到的各种知识之上的路径。 这意味着您可以从初学者成长为专家，并且仍然使用相同的工具——只是方式不同。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ordinary-practitioner",
   "metadata": {},
   "source": [
    "因此，没有一种使用 Keras 的“真正”方式。 相反，Keras 提供了一系列工作流程，从非常简单到非常灵活。 构建 Keras 模型有不同的方法，训练它们的方法也不同，以满足不同的需求。 由于所有这些工作流都基于共享 API，例如层和模型，因此任何工作流中的组件都可以在任何其他工作流中使用：它们都可以相互通信。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "loose-rover",
   "metadata": {},
   "source": [
    "## 构建 Keras 模型的不同方式"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accepting-socket",
   "metadata": {},
   "source": [
    "在 Keras 中构建模型有 3种API：\n",
    "* Sequential 模型，最容易上手的 API——它基本上是一个 Python 列表。 因此，它仅限于简单的层堆叠。\n",
    "* Functional API，专注于类似图的模型架构。 它代表了可用性和灵活性之间的一个很好的中间点，因此，它是最常用的模型构建 API。\n",
    "* 模型子类化，一个低级选项，您可以从头开始自己编写所有内容。 如果您想完全控制每一件小事，这是理想的选择。 但是，您将无法访问许多内置的 Keras 功能，并且更容易犯错误。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mounted-romance",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/008i3skNgy1gsvjgio2rwj31980dedhv.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sophisticated-strip",
   "metadata": {},
   "source": [
    "### 顺序模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "worldwide-colony",
   "metadata": {},
   "source": [
    "构建 Keras 模型的最简单方法是 Sequential 模型，您已经知道："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "professional-smith",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.34 s (started: 2021-07-28 20:38:49 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "model = keras.Sequential([\n",
    " layers.Dense(64, activation=\"relu\"),\n",
    " layers.Dense(10, activation=\"softmax\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "satellite-second",
   "metadata": {},
   "source": [
    "请注意，可以通过方法增量构建相同的模型，类似于 Python 列表的 add() 方法： append()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "found-mouth",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 5.67 ms (started: 2021-07-28 06:42:53 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential()\n",
    "model.add(layers.Dense(64, activation=\"relu\"))\n",
    "model.add(layers.Dense(10, activation=\"softmax\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "convertible-premium",
   "metadata": {},
   "source": [
    "你在第 4 章中已经看到，层只有在第一次被调用时才会被构建（也就是说，创建它们的权重）。 这是因为层权重的形状取决于其输入的形状：在知道输入形状之前，无法创建它们。\n",
    "\n",
    "因此，上面的 Sequential 模型没有任何权重，直到您在某些数据上实际调用它，或者使用输入形状调用其方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "raising-skill",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Weights for model sequential_1 have not yet been created. Weights are created when the Model is first called on inputs or `build()` is called with an `input_shape`.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-dd9f7a7f3f1d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mweights\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36mweights\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   2444\u001b[0m       \u001b[0mA\u001b[0m \u001b[0mlist\u001b[0m \u001b[0mof\u001b[0m \u001b[0mvariables\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2445\u001b[0m     \"\"\"\n\u001b[0;32m-> 2446\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_dedup_weights\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_undeduplicated_weights\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2447\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2448\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36m_undeduplicated_weights\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   2449\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_undeduplicated_weights\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2450\u001b[0m     \u001b[0;34m\"\"\"Returns the undeduplicated list of all layer variables/weights.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2451\u001b[0;31m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_assert_weights_created\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2452\u001b[0m     \u001b[0mweights\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2453\u001b[0m     \u001b[0;32mfor\u001b[0m \u001b[0mlayer\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_self_tracked_trackables\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py\u001b[0m in \u001b[0;36m_assert_weights_created\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    530\u001b[0m     \u001b[0;31m# When the graph has not been initialized, use the Model's implementation to\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    531\u001b[0m     \u001b[0;31m# to check if the weights has been created.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 532\u001b[0;31m     \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunctional\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mFunctional\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_assert_weights_created\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=bad-super-call\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    533\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    534\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36m_assert_weights_created\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   2572\u001b[0m       \u001b[0;31m# been invoked yet, this will cover both sequential and subclass model.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2573\u001b[0m       \u001b[0;31m# Also make sure to exclude Model class itself which has build() defined.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2574\u001b[0;31m       raise ValueError('Weights for model %s have not yet been created. '\n\u001b[0m\u001b[1;32m   2575\u001b[0m                        \u001b[0;34m'Weights are created when the Model is first called on '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2576\u001b[0m                        \u001b[0;34m'inputs or `build()` is called with an `input_shape`.'\u001b[0m \u001b[0;34m%\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: Weights for model sequential_1 have not yet been created. Weights are created when the Model is first called on inputs or `build()` is called with an `input_shape`."
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.11 s (started: 2021-07-27 15:31:37 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "residential-manor",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Variable 'dense_2/kernel:0' shape=(3, 64) dtype=float32, numpy=\n",
       " array([[-0.00819355,  0.15351084, -0.23104338, -0.09885751,  0.2628876 ,\n",
       "         -0.20201147,  0.08822173,  0.28065652, -0.25919423,  0.14183763,\n",
       "         -0.07550877,  0.26725054, -0.13888718, -0.16665961,  0.02244398,\n",
       "         -0.13757768, -0.23935236, -0.14157756, -0.10289697, -0.23451564,\n",
       "          0.00623435, -0.19813038, -0.21516216,  0.23319238,  0.11614984,\n",
       "         -0.04733825,  0.26974314,  0.1734513 ,  0.1687446 ,  0.15775165,\n",
       "          0.10909671, -0.0678076 ,  0.22353363, -0.23721272,  0.05471078,\n",
       "          0.21833688,  0.2523135 ,  0.25877118,  0.14999378, -0.14234525,\n",
       "          0.27807528,  0.0214887 ,  0.02348122,  0.16687799,  0.11501056,\n",
       "         -0.23760428, -0.22343895,  0.10037541,  0.0794273 , -0.02018911,\n",
       "         -0.2630825 ,  0.07161856, -0.05101891, -0.14118814, -0.2773213 ,\n",
       "         -0.0763635 , -0.20710817,  0.11496246,  0.17371288,  0.08795267,\n",
       "          0.23122561,  0.03001878,  0.1955021 , -0.20412529],\n",
       "        [-0.13369545,  0.19753414, -0.20661381, -0.09781598, -0.22857697,\n",
       "         -0.2623064 , -0.02784389, -0.11103879,  0.161715  , -0.15945116,\n",
       "          0.08133   , -0.13229547, -0.2353554 ,  0.04065776,  0.21883023,\n",
       "          0.16849345,  0.22944206,  0.11634633, -0.02950194, -0.18885386,\n",
       "         -0.26076058, -0.27410728,  0.05156943, -0.00702822, -0.17518227,\n",
       "         -0.18928787, -0.03966531,  0.20667517,  0.01368222, -0.24020511,\n",
       "          0.11079392,  0.22974783, -0.14171119, -0.2554918 ,  0.02940828,\n",
       "         -0.18632653, -0.2888427 ,  0.20798433, -0.00067738, -0.13281317,\n",
       "         -0.2652368 , -0.13592912, -0.21478173, -0.27883047,  0.12214765,\n",
       "          0.15452006, -0.2789494 , -0.17775783,  0.20106518,  0.0038009 ,\n",
       "          0.04941365, -0.08153941, -0.251687  , -0.25184423, -0.14357172,\n",
       "          0.09127942,  0.15368772, -0.2821014 , -0.12771396,  0.28334683,\n",
       "         -0.05190161, -0.20599422,  0.19908929, -0.25130326],\n",
       "        [-0.09376693,  0.22877932, -0.28857073, -0.2083748 ,  0.25640476,\n",
       "          0.24461645, -0.0393205 ,  0.23994505,  0.18572173,  0.08301893,\n",
       "          0.08012909,  0.19337642,  0.27213883,  0.12910601,  0.1718607 ,\n",
       "         -0.07985276,  0.0559966 ,  0.05280408,  0.00633702, -0.00749269,\n",
       "          0.0692862 ,  0.18296534,  0.10116795, -0.13931613,  0.1857506 ,\n",
       "          0.022594  ,  0.17873901, -0.03594998,  0.23433673, -0.1600111 ,\n",
       "         -0.13541   , -0.14079566,  0.1693295 , -0.03988135, -0.09047417,\n",
       "          0.16849315, -0.13326587,  0.27419758, -0.2373248 , -0.21318355,\n",
       "         -0.13335855, -0.2005178 , -0.14119963, -0.17835438, -0.11188133,\n",
       "          0.07186171,  0.17287838,  0.06411737,  0.13466585,  0.21690977,\n",
       "          0.11099905,  0.28330892,  0.04141349, -0.0302518 ,  0.07306355,\n",
       "         -0.06239696,  0.20916212,  0.03124726,  0.0471015 ,  0.17859253,\n",
       "         -0.28879413, -0.07176396, -0.15992184, -0.05487859]],\n",
       "       dtype=float32)>,\n",
       " <tf.Variable 'dense_2/bias:0' shape=(64,) dtype=float32, numpy=\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)>,\n",
       " <tf.Variable 'dense_3/kernel:0' shape=(64, 10) dtype=float32, numpy=\n",
       " array([[-8.91924500e-02,  4.56291139e-02, -1.89759731e-01,\n",
       "          2.39290386e-01, -2.27863893e-01, -1.14050642e-01,\n",
       "         -2.75688559e-01, -2.05490291e-01,  2.80789703e-01,\n",
       "         -2.63640672e-01],\n",
       "        [-1.69637948e-01,  2.27270991e-01,  4.25577164e-05,\n",
       "         -6.05481863e-03,  9.42659378e-03,  2.20839947e-01,\n",
       "         -1.71857372e-01,  1.68888241e-01, -2.02996463e-01,\n",
       "         -1.53600574e-01],\n",
       "        [ 1.57189339e-01, -9.58677828e-02,  2.79036611e-01,\n",
       "          2.90482938e-02,  2.14302450e-01,  5.25195301e-02,\n",
       "         -2.73088932e-01,  2.05788821e-01,  1.66537941e-01,\n",
       "          2.65841097e-01],\n",
       "        [-2.81370997e-01,  9.58265662e-02, -1.84987456e-01,\n",
       "         -1.05841771e-01, -1.59314469e-01, -4.13202792e-02,\n",
       "          8.39318633e-02, -2.35858917e-01,  8.28180015e-02,\n",
       "          1.99627817e-01],\n",
       "        [-1.03224441e-01,  9.77571905e-02,  2.47792155e-01,\n",
       "         -7.26356655e-02,  2.28163868e-01,  1.34173930e-02,\n",
       "          4.45052981e-03,  1.82119012e-01, -1.08210087e-01,\n",
       "          7.70245492e-02],\n",
       "        [ 8.14032555e-02, -8.97840410e-02,  7.01917410e-02,\n",
       "         -8.51377845e-03, -2.13223398e-01,  1.72451824e-01,\n",
       "          1.73188627e-01, -2.65212387e-01,  3.65292132e-02,\n",
       "          1.17734373e-01],\n",
       "        [-1.79274470e-01, -1.73907623e-01,  3.68835926e-02,\n",
       "         -6.79498315e-02, -1.56604603e-01,  1.13812774e-01,\n",
       "         -1.66560128e-01,  2.68913776e-01, -2.35342205e-01,\n",
       "          1.63759559e-01],\n",
       "        [ 2.49524444e-01, -2.72668898e-02,  1.27218485e-01,\n",
       "         -2.83781886e-01,  2.66693801e-01, -1.82648748e-01,\n",
       "         -1.23149887e-01, -1.91947848e-01, -2.68279880e-01,\n",
       "         -2.05311611e-01],\n",
       "        [-1.58019364e-02,  1.06287479e-01, -2.58452058e-01,\n",
       "         -1.25623763e-01, -2.08819106e-01, -2.64574349e-01,\n",
       "         -5.44764251e-02,  8.52116346e-02, -2.84268081e-01,\n",
       "         -2.52203941e-01],\n",
       "        [ 6.78783953e-02,  2.84080535e-01, -2.55712926e-01,\n",
       "          5.60449958e-02,  5.11393547e-02,  1.51845455e-01,\n",
       "         -6.06630892e-02, -1.24537408e-01,  2.78382599e-02,\n",
       "         -2.73452342e-01],\n",
       "        [-1.36995733e-01,  2.18580872e-01, -1.82416439e-02,\n",
       "          6.23050630e-02, -2.00530052e-01,  1.96510673e-01,\n",
       "          9.12619829e-02,  1.30456388e-02, -1.47706866e-02,\n",
       "          3.61421704e-03],\n",
       "        [-7.81039298e-02, -1.80455253e-01, -2.83161432e-01,\n",
       "          2.38224894e-01, -6.09342456e-02,  1.86641246e-01,\n",
       "          1.29309744e-01,  2.19493777e-01, -1.13036454e-01,\n",
       "         -4.01307195e-02],\n",
       "        [-2.06611216e-01,  1.35575026e-01, -2.21737385e-01,\n",
       "         -7.49552399e-02, -1.83921605e-01, -4.68224734e-02,\n",
       "          3.53372097e-02, -1.47691175e-01, -4.39461619e-02,\n",
       "         -3.26289833e-02],\n",
       "        [-2.84567475e-01, -1.64820135e-01, -7.36116469e-02,\n",
       "         -2.20658422e-01, -1.41214564e-01, -1.36537209e-01,\n",
       "         -3.82493734e-02,  1.18030638e-01,  1.47886008e-01,\n",
       "         -6.95893466e-02],\n",
       "        [ 1.08658075e-01, -4.22155261e-02, -1.54175937e-01,\n",
       "          1.80102170e-01, -1.10540777e-01, -1.35844186e-01,\n",
       "          7.27879405e-02, -2.06491530e-01, -2.34159246e-01,\n",
       "          7.27099478e-02],\n",
       "        [-1.11806437e-01,  1.98635817e-01, -6.37454540e-02,\n",
       "          2.18972772e-01, -5.36991656e-02, -4.39334661e-02,\n",
       "          1.24299318e-01, -1.27918139e-01, -1.82597429e-01,\n",
       "          8.69984925e-02],\n",
       "        [-1.27010122e-01,  1.81372702e-01, -1.66446552e-01,\n",
       "         -1.05070755e-01,  2.09807217e-01, -2.03662112e-01,\n",
       "         -6.06478155e-02,  2.13992923e-01, -1.64194122e-01,\n",
       "         -2.54737169e-01],\n",
       "        [-2.58799702e-01,  2.51685768e-01,  2.78054863e-01,\n",
       "          9.37753618e-02,  1.45378530e-01, -1.82767287e-01,\n",
       "          2.58524746e-01, -2.32622713e-01, -1.94756091e-01,\n",
       "          2.63389975e-01],\n",
       "        [-8.51364285e-02, -2.28346586e-01,  1.96057171e-01,\n",
       "          3.95154357e-02,  2.02825308e-01, -4.93928343e-02,\n",
       "          6.90640807e-02, -2.44635880e-01, -2.76459754e-01,\n",
       "         -2.91029513e-02],\n",
       "        [-3.22057009e-02, -2.24723682e-01,  1.85290277e-02,\n",
       "          2.33561724e-01, -7.33797401e-02, -1.38368845e-01,\n",
       "          5.64927459e-02, -7.22237825e-02,  2.64991492e-01,\n",
       "         -2.34166026e-01],\n",
       "        [-1.98031127e-01, -2.78030932e-02, -1.81627899e-01,\n",
       "          9.19912457e-02, -1.40346870e-01, -2.71150082e-01,\n",
       "          2.20384508e-01,  1.60908639e-01, -8.20639729e-02,\n",
       "          6.40958250e-02],\n",
       "        [-1.13678545e-01,  3.99180353e-02, -3.46890092e-02,\n",
       "          6.90082312e-02, -7.52554387e-02, -2.62631238e-01,\n",
       "         -1.12546906e-01, -1.98324740e-01, -2.47842222e-01,\n",
       "         -2.61488736e-01],\n",
       "        [ 2.89965868e-02, -3.34151387e-02, -2.15076089e-01,\n",
       "          1.06244296e-01,  7.80595243e-02, -1.13398433e-02,\n",
       "         -2.76044011e-03, -7.14692622e-02,  1.26726419e-01,\n",
       "          1.69434279e-01],\n",
       "        [-2.30611086e-01,  2.18570828e-02,  1.30730778e-01,\n",
       "         -1.39970019e-01,  1.22996598e-01, -2.71510124e-01,\n",
       "         -1.28325477e-01,  1.98865533e-01, -2.78430521e-01,\n",
       "         -1.13978550e-01],\n",
       "        [ 1.88726187e-02, -2.37629592e-01,  1.09799802e-02,\n",
       "         -5.71040064e-02,  1.14554107e-01,  8.40315819e-02,\n",
       "          7.57841468e-02,  5.26264608e-02, -1.20011568e-02,\n",
       "         -2.63989210e-01],\n",
       "        [ 7.06860423e-02,  1.71270132e-01, -1.65189117e-01,\n",
       "          1.39540493e-01, -1.03276312e-01, -5.28232008e-02,\n",
       "          1.13462448e-01, -3.54952514e-02, -2.60061920e-01,\n",
       "         -3.72273028e-02],\n",
       "        [-2.51949221e-01,  1.04472250e-01,  2.06877083e-01,\n",
       "         -1.79298222e-01, -1.98483810e-01,  2.16957420e-01,\n",
       "         -1.72871947e-02,  5.83844781e-02, -8.26855600e-02,\n",
       "          9.14258659e-02],\n",
       "        [-1.35780171e-01, -2.68309265e-01,  1.28354728e-01,\n",
       "         -1.19918168e-01, -2.44635954e-01,  1.12388581e-01,\n",
       "         -2.30293497e-01, -6.52559251e-02, -2.68072009e-01,\n",
       "          1.36843920e-02],\n",
       "        [-1.86934441e-01,  4.19866145e-02,  2.11146802e-01,\n",
       "          1.85699403e-01,  5.96480072e-02,  8.59234631e-02,\n",
       "          4.56023216e-03,  2.48742133e-01, -2.18038857e-01,\n",
       "         -7.65337199e-02],\n",
       "        [-4.04497981e-03, -3.86103392e-02, -2.38358647e-01,\n",
       "          1.91414237e-01,  1.45854950e-02,  1.83009505e-01,\n",
       "          7.22609162e-02,  7.39149153e-02,  2.43868381e-01,\n",
       "          1.76288426e-01],\n",
       "        [-5.51086813e-02, -1.05420724e-01, -2.78128058e-01,\n",
       "         -1.92037344e-01, -2.70671397e-01,  1.10451996e-01,\n",
       "          2.03795433e-02, -1.09437242e-01, -1.08309686e-02,\n",
       "          2.78206497e-01],\n",
       "        [-1.74158201e-01, -1.16626695e-01,  2.58227199e-01,\n",
       "          2.76773602e-01, -2.09820673e-01, -9.68012512e-02,\n",
       "          2.84337699e-02,  1.97768211e-03,  1.58019811e-01,\n",
       "          4.36276197e-02],\n",
       "        [-5.91476113e-02,  2.45515972e-01, -2.26347923e-01,\n",
       "          3.88415754e-02,  2.86070406e-02,  2.33657956e-02,\n",
       "          6.03463948e-02,  2.60449141e-01,  1.56388909e-01,\n",
       "          2.09958285e-01],\n",
       "        [-2.25961506e-01,  2.53690273e-01, -2.67639488e-01,\n",
       "          1.58093899e-01, -2.67902702e-01,  2.43146271e-01,\n",
       "         -1.27634645e-01,  1.11742556e-01,  2.03646570e-01,\n",
       "         -1.60911977e-02],\n",
       "        [-2.44669139e-01,  2.19875604e-01,  1.60669863e-01,\n",
       "         -4.07273322e-02, -5.82448840e-02,  1.59151465e-01,\n",
       "          1.89187288e-01,  1.00700796e-01, -1.83460638e-01,\n",
       "          1.30529910e-01],\n",
       "        [-2.41627723e-01,  3.39294672e-02, -7.75427520e-02,\n",
       "         -1.61904305e-01,  1.10881925e-01, -2.53015578e-01,\n",
       "         -6.55975342e-02,  1.24709427e-01,  2.79615849e-01,\n",
       "          1.54869020e-01],\n",
       "        [-1.00974604e-01, -2.54841238e-01,  7.06529021e-02,\n",
       "         -1.81536466e-01, -3.65874171e-03,  1.53337449e-01,\n",
       "         -1.81774676e-01,  8.78991783e-02,  9.64571834e-02,\n",
       "         -1.11626595e-01],\n",
       "        [-6.37409836e-02, -4.36578244e-02, -1.20609686e-01,\n",
       "         -2.43231192e-01, -1.59655541e-01,  1.34114325e-01,\n",
       "         -1.34756744e-01, -1.00013286e-01,  2.50276893e-01,\n",
       "         -8.97598714e-02],\n",
       "        [ 2.55998284e-01,  2.32704848e-01,  5.75599372e-02,\n",
       "          6.15412295e-02,  7.62065649e-02,  1.19859993e-01,\n",
       "          2.59828597e-01, -1.06651217e-01, -1.15921810e-01,\n",
       "         -5.52767068e-02],\n",
       "        [-1.04859561e-01,  2.46757537e-01, -1.64519519e-01,\n",
       "          2.67405957e-01, -1.85973346e-01,  4.83952761e-02,\n",
       "          1.16362065e-01, -1.87453181e-01, -1.28393233e-01,\n",
       "         -2.07808301e-01],\n",
       "        [ 6.98306859e-02, -1.22726530e-01,  2.61171550e-01,\n",
       "         -1.49330974e-01,  5.87106049e-02,  3.03224623e-02,\n",
       "          2.12165296e-01,  6.93858862e-02,  2.32184201e-01,\n",
       "         -1.51426360e-01],\n",
       "        [-1.48907810e-01, -2.67323464e-01, -8.07851553e-03,\n",
       "         -3.18318605e-03,  2.26824850e-01,  1.01196200e-01,\n",
       "         -9.01120752e-02,  2.19171017e-01, -1.87228680e-01,\n",
       "          2.31257409e-01],\n",
       "        [ 5.53820729e-02, -1.27512366e-01, -2.74422824e-01,\n",
       "         -1.10363051e-01, -1.56196117e-01,  5.82061112e-02,\n",
       "          1.28136069e-01, -1.26971021e-01, -4.77211326e-02,\n",
       "         -5.44069856e-02],\n",
       "        [-1.59465998e-01, -8.40264410e-02, -2.67505527e-01,\n",
       "         -1.58704221e-02,  2.32310444e-01,  1.63059890e-01,\n",
       "          1.31682783e-01, -1.97045505e-01,  1.77695781e-01,\n",
       "         -1.30568266e-01],\n",
       "        [ 2.15357810e-01, -1.53412223e-02,  2.57045031e-02,\n",
       "          1.22066855e-01,  2.16104239e-01, -2.00399697e-01,\n",
       "         -4.23961878e-03, -5.53332567e-02, -2.72300482e-01,\n",
       "          1.10170662e-01],\n",
       "        [-2.12063789e-01,  2.10590661e-01,  1.81653351e-01,\n",
       "         -4.04111743e-02,  1.04748756e-01,  1.84985489e-01,\n",
       "          2.18594640e-01, -2.33082861e-01, -7.90653676e-02,\n",
       "          1.95981950e-01],\n",
       "        [ 2.82525271e-01, -1.93211541e-01,  2.38923818e-01,\n",
       "          7.12129474e-03,  1.84561461e-01,  1.27748162e-01,\n",
       "         -2.13828772e-01, -1.90356195e-01,  2.83327848e-01,\n",
       "          1.57357216e-01],\n",
       "        [ 2.34650284e-01,  1.32431418e-01, -2.05023229e-01,\n",
       "          2.13695973e-01,  7.69684911e-02, -6.16673678e-02,\n",
       "          1.34436369e-01, -1.45891160e-01, -1.48434088e-01,\n",
       "         -4.53911722e-02],\n",
       "        [ 1.64109468e-02,  2.73364782e-02, -1.58620641e-01,\n",
       "          2.23726124e-01, -5.18015027e-03, -5.95176220e-03,\n",
       "         -2.74073124e-01, -1.02571219e-01,  2.40746170e-01,\n",
       "         -9.32584703e-02],\n",
       "        [-1.59578830e-01, -2.40342915e-02,  2.64726728e-01,\n",
       "         -5.83871752e-02,  5.36164641e-02, -1.15033820e-01,\n",
       "         -1.63724944e-01, -2.41599068e-01,  7.06061125e-02,\n",
       "          2.26332575e-01],\n",
       "        [-2.47910038e-01,  2.84864008e-02,  9.19222832e-04,\n",
       "         -8.53846222e-02,  9.78781283e-02,  2.71605819e-01,\n",
       "          1.71320856e-01, -2.27677405e-01, -7.82597065e-03,\n",
       "          6.50056303e-02],\n",
       "        [ 2.17351764e-01, -1.70100138e-01, -1.12341955e-01,\n",
       "          1.16429478e-01,  1.35347039e-01, -1.49267152e-01,\n",
       "         -2.68846750e-04,  2.57448852e-02, -2.12287426e-01,\n",
       "          2.06970751e-01],\n",
       "        [-1.89396992e-01, -2.90474296e-02, -2.36822188e-01,\n",
       "         -8.94746631e-02,  1.47242576e-01,  1.83008820e-01,\n",
       "         -1.88112319e-01,  2.15478748e-01, -2.76973754e-01,\n",
       "          2.50751108e-01],\n",
       "        [ 2.82371908e-01, -2.63374209e-01,  1.45247161e-01,\n",
       "          2.59672850e-01,  2.44078308e-01, -2.53750801e-01,\n",
       "          1.71632797e-01,  2.30025381e-01,  2.77039558e-01,\n",
       "          7.97930658e-02],\n",
       "        [-3.54340822e-02, -2.42995411e-01, -2.48198971e-01,\n",
       "          1.65185988e-01,  1.75810486e-01, -6.48642033e-02,\n",
       "          1.73512101e-02, -2.04697549e-01,  1.28614694e-01,\n",
       "          2.59758443e-01],\n",
       "        [ 1.94519490e-01,  2.25199610e-01,  1.18802279e-01,\n",
       "         -1.50944620e-01,  2.63544917e-03, -3.17453444e-02,\n",
       "          7.46335685e-02, -2.27610394e-01,  2.58446932e-02,\n",
       "         -2.68032014e-01],\n",
       "        [-2.77984887e-01, -1.44553065e-01,  1.91513777e-01,\n",
       "          1.85636550e-01, -1.34697556e-03, -1.83857858e-01,\n",
       "         -2.15344250e-01,  5.67457080e-03,  2.49669760e-01,\n",
       "          2.28701830e-02],\n",
       "        [-1.49562135e-01, -3.76988053e-02,  1.55827284e-01,\n",
       "          2.07407892e-01, -2.73966789e-03,  1.22272342e-01,\n",
       "         -1.90238059e-02, -1.89183652e-02,  1.16035104e-01,\n",
       "          2.55590230e-01],\n",
       "        [ 1.35668367e-01, -2.23279476e-01, -2.62420446e-01,\n",
       "          1.20776623e-01,  1.58906728e-01, -5.96400052e-02,\n",
       "          2.02511877e-01, -6.09072149e-02, -1.87116593e-01,\n",
       "          2.23792106e-01],\n",
       "        [-1.58908144e-01, -9.96543020e-02,  1.84714556e-01,\n",
       "         -1.42720342e-01,  1.21759176e-02, -6.74047470e-02,\n",
       "         -1.54726863e-01,  1.57149017e-01, -1.85754806e-01,\n",
       "         -9.58705693e-02],\n",
       "        [-4.08533365e-02, -1.29062623e-01,  2.26216108e-01,\n",
       "          6.09158576e-02, -1.03592873e-04,  2.65767962e-01,\n",
       "          9.22630131e-02, -2.69028842e-01,  1.63462341e-01,\n",
       "          2.27854103e-01],\n",
       "        [-2.36701488e-01,  7.51479566e-02,  2.22071916e-01,\n",
       "         -2.66945601e-01, -8.34136754e-02, -8.49176794e-02,\n",
       "          4.64320481e-02,  2.58314043e-01,  2.86140144e-02,\n",
       "         -7.43034333e-02],\n",
       "        [-1.67850152e-01, -1.26170546e-01,  7.88982213e-02,\n",
       "          1.16214484e-01, -1.40582576e-01, -1.84496701e-01,\n",
       "         -8.09801221e-02, -1.84454665e-01,  1.11282676e-01,\n",
       "          2.45812207e-01],\n",
       "        [ 1.96098179e-01, -3.17496955e-02,  1.56612545e-01,\n",
       "         -1.00390151e-01,  3.71001661e-02, -2.02197403e-01,\n",
       "         -1.89145267e-01, -1.88714504e-01,  7.11645782e-02,\n",
       "         -2.01672763e-01]], dtype=float32)>,\n",
       " <tf.Variable 'dense_3/bias:0' shape=(10,) dtype=float32, numpy=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)>]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 227 ms (started: 2021-07-27 15:33:48 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 构建模型——现在模型将需要 shape (3,)的样本。 None输入形状表明批量大小可以是任何东西。\n",
    "model.build(input_shape=(None, 3))\n",
    "model.weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "demanding-snapshot",
   "metadata": {},
   "source": [
    "模型构建完成后，您可以通过方法显示其内容，该方法提供的 summary() 便于调试："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "adjacent-navigator",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_2 (Dense)              (None, 64)                256       \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 906\n",
      "Trainable params: 906\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 2.52 ms (started: 2021-07-27 15:34:21 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sonic-sewing",
   "metadata": {},
   "source": [
    "如您所见，您的模型恰好被命名为“sequential_1”。 实际上，您可以为 Keras 中的所有内容命名——每个模型、每个层。 像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "female-academy",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"my_example_model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "my_first_layer (Dense)       (None, 64)                256       \n",
      "_________________________________________________________________\n",
      "my_last_layer (Dense)        (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 906\n",
      "Trainable params: 906\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 220 ms (started: 2021-07-28 06:43:06 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential(name=\"my_example_model\")\n",
    "model.add(layers.Dense(64, activation=\"relu\", name=\"my_first_layer\"))\n",
    "model.add(layers.Dense(10, activation=\"softmax\", name=\"my_last_layer\"))\n",
    "model.build((None, 3))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "monetary-india",
   "metadata": {},
   "source": [
    "在逐步构建 Sequential 模型时，能够在添加每一层后打印当前模型外观的摘要非常有用。 但是在构建模型之前您无法打印摘要！ 实际上有一种方法可以让您的 Sequential 即时构建：只需提前声明模型输入的形状。 您可以通过 Input 类执行此操作："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "dynamic-excuse",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 34.5 ms (started: 2021-07-27 15:39:42 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.Sequential()\n",
    "#使用 Input 来声明输入的形状。 请注意，参数 Input shape 必须是每个样本的形状，而不是一批的形状。\n",
    "model.add(layers.InputLayer(input_shape=(4,))) \n",
    "model.add(layers.Dense(64, activation=\"relu\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "enclosed-appendix",
   "metadata": {},
   "source": [
    "现在，您可以跟踪模型的输出形状如何随着您添加 summary() 更多层而发生变化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "cloudy-measure",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_7\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_5 (Dense)              (None, 64)                320       \n",
      "=================================================================\n",
      "Total params: 320\n",
      "Trainable params: 320\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 3.05 ms (started: 2021-07-27 15:40:16 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "spare-quarterly",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_7\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_5 (Dense)              (None, 64)                320       \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 970\n",
      "Trainable params: 970\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 21.2 ms (started: 2021-07-27 15:40:46 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.add(layers.Dense(10, activation=\"softmax\"))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "comic-tunisia",
   "metadata": {},
   "source": [
    "在处理以复杂方式转换输入的层时，这是一个非常常见的调试工作流程，例如您将在第 8 章中学习的卷积层。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daily-northeast",
   "metadata": {},
   "source": [
    "### 函数型API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "instrumental-result",
   "metadata": {},
   "source": [
    "Sequential 模型易于使用，但其适用性极其有限：它只能表达具有单个输入和单个输出的模型，以顺序方式一层接一层地应用。 在实践中，经常会遇到具有多个输入（例如，图像及其元数据）、多个输出（您想要预测数据的不同事物）或非线性拓扑的模型。\n",
    "\n",
    "在这种情况下，您可以使用 Functional API 构建模型。 这是您在野外使用时会遇到的大多数 Keras 模型。 它既有趣又强大——感觉就像在玩乐高积木。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "complicated-translator",
   "metadata": {},
   "source": [
    "**一个简单的例子** "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "controlling-filling",
   "metadata": {},
   "source": [
    "让我们从简单的事情开始：我们在上一节中使用的两层堆栈。 它的功能 API 版本如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "severe-reading",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 35.3 ms (started: 2021-07-27 15:54:47 +08:00)\n"
     ]
    }
   ],
   "source": [
    "inputs = layers.Input(shape=(3,), name=\"my_input\")\n",
    "features = layers.Dense(64, activation=\"relu\")(inputs)\n",
    "outputs = layers.Dense(10, activation=\"softmax\")(features)\n",
    "model = keras.Model(inputs=inputs, outputs=outputs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "egyptian-theta",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "my_input (InputLayer)        [(None, 3)]               0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 64)                256       \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 906\n",
      "Trainable params: 906\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 2.38 ms (started: 2021-07-27 15:54:54 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "surrounded-protection",
   "metadata": {},
   "source": [
    "让我们一步一步来。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "final-smart",
   "metadata": {},
   "source": [
    "我们首先声明了一个Input（请注意，您也可以为这些输入对象命名，例如输入其他所有内容）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "unlike-parliament",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.66 ms (started: 2021-07-27 16:05:41 +08:00)\n"
     ]
    }
   ],
   "source": [
    "inputs = layers.Input(shape=(3,), name=\"my_input\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unique-worship",
   "metadata": {},
   "source": [
    "此对象保存有关模型将输入处理的数据的形状和数据类型的信息："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "physical-associate",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([None, 3])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4.63 ms (started: 2021-07-27 16:06:07 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 该模型将处理每个样本具有 shape 的批次。 每批次的 (3,) 样本数量是可变的（由批次大小表示）。\n",
    "inputs.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "black-debate",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tf.float32"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.85 ms (started: 2021-07-27 16:08:45 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 这些批次将具有 dtype float32\n",
    "inputs.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "addressed-publicity",
   "metadata": {},
   "source": [
    "我们称这样的对象为“符号张量”。 它不包含任何实际数据，但它对模型在使用时将看到的实际数据张量的规格进行编码。 它代表未来的数据张量。\n",
    "\n",
    "接下来，我们创建了一个层并在输入上调用它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "fresh-willow",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 31.7 ms (started: 2021-07-27 16:09:43 +08:00)\n"
     ]
    }
   ],
   "source": [
    "features = layers.Dense(64, activation=\"relu\")(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "outstanding-fighter",
   "metadata": {},
   "source": [
    "所有 Keras 层都可以在数据的真实张量或这些符号张量上调用。 在后一种情况下，它们返回一个新的符号张量，带有更新的形状和 dtype 信息："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "thirty-expense",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tf.float32"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.46 ms (started: 2021-07-27 16:10:28 +08:00)\n"
     ]
    }
   ],
   "source": [
    "features.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "viral-observation",
   "metadata": {},
   "source": [
    "获得最终输出后，我们通过在构造函数中指定其输入和输出来实例化模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "chief-marathon",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 35.1 ms (started: 2021-07-27 16:11:02 +08:00)\n"
     ]
    }
   ],
   "source": [
    "outputs = layers.Dense(10, activation=\"softmax\")(features)\n",
    "model = keras.Model(inputs=inputs, outputs=outputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "falling-individual",
   "metadata": {},
   "source": [
    "这是我们模型的摘要："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "swedish-subsection",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "my_input (InputLayer)        [(None, 3)]               0         \n",
      "_________________________________________________________________\n",
      "dense_9 (Dense)              (None, 64)                256       \n",
      "_________________________________________________________________\n",
      "dense_10 (Dense)             (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 906\n",
      "Trainable params: 906\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 3.49 ms (started: 2021-07-27 16:11:47 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "apart-delivery",
   "metadata": {},
   "source": [
    "**多输入多输出型号**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "waiting-sigma",
   "metadata": {},
   "source": [
    "与这个玩具模型不同，大多数深度学习模型看起来不像列表——它们看起来像图表。 例如，它们可能有多个输入或多个输出。 正是对于这种模型，Functional API 才真正闪耀。\n",
    "\n",
    "假设您正在构建一个系统来按优先级对客户支持票进行排名，并将它们路由到适当的部门。 您的模型具有三个输入：\n",
    "* 票的标题（文本输入）\n",
    "* 工单正文（文本输入）\n",
    "* 用户添加的任何标签（分类输入，此处假定为单热编码）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "national-vinyl",
   "metadata": {},
   "source": [
    "我们可以将文本输入编码为大小为词汇表大小的 1 和 0 的数组（有关文本编码技术的详细信息，请参见第 11 章）。\n",
    "\n",
    "您的模型还有两个输出：\n",
    "* 票证的优先级分数，一个介于 0 和 1 之间的标量（sigmoid 输出）\n",
    "* 应该处理工单的部门（部门集上的 softmax）\n",
    "\n",
    "您可以使用 Functional API 用几行代码构建此模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "relative-grade",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 58.6 ms (started: 2021-07-27 16:16:43 +08:00)\n"
     ]
    }
   ],
   "source": [
    "vocabulary_size = 10000\n",
    "num_tags = 100\n",
    "num_departments = 4\n",
    "\n",
    "# 定义模型输入\n",
    "title = keras.Input(shape=(vocabulary_size,), name=\"title\")\n",
    "text_body = keras.Input(shape=(vocabulary_size,), name=\"text_body\") \n",
    "tags = keras.Input(shape=(num_tags,), name=\"tags\")\n",
    "\n",
    "# 将输入特征组合成单个张量 ，通过将它们连接起来\n",
    "features = layers.Concatenate()([title, text_body, tags])\n",
    "# 应用中间层将输入特征重新组合成更丰富的表示\n",
    "features = layers.Dense(64, activation=\"relu\")(features)\n",
    "\n",
    "# 定义模型输出\n",
    "priority = layers.Dense(1, activation=\"sigmoid\", name=\"priority\")(features) \n",
    "department = layers.Dense(num_departments, activation=\"softmax\", name=\"department\")(features) \n",
    "\n",
    "# 通过指定其输入和输出来创建模型\n",
    "model = keras.Model(inputs=[title, text_body, tags], outputs=[priority, department])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "corresponding-usage",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_2\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "title (InputLayer)              [(None, 10000)]      0                                            \n",
      "__________________________________________________________________________________________________\n",
      "text_body (InputLayer)          [(None, 10000)]      0                                            \n",
      "__________________________________________________________________________________________________\n",
      "tags (InputLayer)               [(None, 100)]        0                                            \n",
      "__________________________________________________________________________________________________\n",
      "concatenate (Concatenate)       (None, 20100)        0           title[0][0]                      \n",
      "                                                                 text_body[0][0]                  \n",
      "                                                                 tags[0][0]                       \n",
      "__________________________________________________________________________________________________\n",
      "dense_11 (Dense)                (None, 64)           1286464     concatenate[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "priority (Dense)                (None, 1)            65          dense_11[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "department (Dense)              (None, 4)            260         dense_11[0][0]                   \n",
      "==================================================================================================\n",
      "Total params: 1,286,789\n",
      "Trainable params: 1,286,789\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "time: 2.7 ms (started: 2021-07-27 16:16:50 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "exceptional-estate",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 383 µs (started: 2021-07-27 16:17:10 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.utils import plot_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "august-relations",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApEAAAFgCAIAAABsb7W9AAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVwTd/4/8E9CQjgNR5EbAevRA9GiW1AoUkS0iiAF0VbRikprrdda0bZ2faz9dkGtLrZUxbZrba2AflsUpVa85VoRRbwAjyJyiiAIAuHI/P6Y784vGyAECJlMeD3/IpMPM++ZfDKvzMwnGR5FUQQAAAA0Hp/tAgAAAEApyGwAAABuQGYDAABwAzIbAACAGwSyD7Kysnbs2MFWKaD5Dh8+3P+Z7NixIysrq//zAWDL2rVrPTw8+jkT7G9BGR4eHmvXrmUe/tdx9qNHj44cOaL2koADSktLVdU3srKysrOzVTIrAPU7cuTIo0eP+j8f7G+hR9nZ2XJHOILOjVRyLAVaJikpKSwsTFVzc3d3RzcDjuLxeCqcG94IoEBoaKjcFFzPBgAA4AZkNgAAADcgswEAALgBmQ0AAMANyGwAAABuQGYDAABwAzIbAACAG5DZAAAA3IDMBgAA4AZkNgAAADcgswEAALgBmQ0AAMANyGwAAABuYCGzt2/fzuPxeDyenZ2daudsZGTEk7F9+3bVzr8/NLk26NEAddqEhAR6tnp6eiqcLdHs/qbJtUFvafKrqcm19ZkqM7uxsXHEiBEzZ85U3GzdunUURbm6uqpw0UwB165dI4QEBgZSFLVu3TqVL6LPNLk26NEAddq5c+dSFOXr66va2RLN7m+aXBv0lia/mppcW5/1MbONjIw8PT3lJlIUJZVKpVJpjy212yBc5QEycFsSr5F6YDurBDZjjwbPJhKocF7Gxsb3799X4QwBAACAgTFoAAAA3NDrzKYH4zx//jwjI4O+sC8QCAghycnJzKX+lpYWBS27U11dvXLlSkdHR11dXQsLi+Dg4Ly8vD6vGEO2sOLi4rCwMBMTE3Nz85kzZzJnBWRHGOXk5Pj6+hobGxsYGPj4+GRkZNBtvvjiC7oNcwbm5MmT9JQXXnhB8cZRXnt7e2Jiop+fn5WVlb6+vouLS2xsLH25oa6uTnY8xRdffEG3Z6aEhITQM1GwJWW3RmFh4Zw5c8zNzemHT5486deGVinFW1LBCnp6ejIrOH/+fELIlClTmCl1dXX9f40KCgpmzJghFovlegitpqZm7dq1w4cP19XVNTU1nT59+rlz5+T+PSgoSCwWGxoaenl5paenM08p/xL3Dd4LnHsvKNiMCrYPg+lsBgYGf/nLX44fP868HZYsWUIIkUgkn3/++ejRow0MDMzMzAICAo4dO9bR0dHPstHTBrCnUTISExPlpnTH0NBw0qRJnacHBgYSQpqbm3ts6erqamtryzwsLy8fNmyYpaXliRMnGhoabt686e3traenl5mZybTx8fExMzPLyspSUJjsiIPOhQUGBmZmZjY2Nqalpenr60+YMEGuJENDQw8PD7pNTk7OmDFjdHV1z58/r2B13NzczM3Nldk43dUmKyUlhRDy5Zdf1tbWVldX79q1i8/n0wOgaP7+/nw+/969e7L/5eHhcfDgQfpvZbYkvTW8vb3PnTv3/Pnz7OxsHR2d6upqBYUp3zd6FBISEhISokzLLrdkjyuYl5dnaGjo6ura2NhIUVRLS8vrr79+6NChHufcI1dXV7FY7OPjk56e3tDQ0LmHVFRUODk5WVpapqSk1NfXFxYWBgcH83i8ffv20Q3u3r1rYmJia2t76tSphoaG/Pz8qVOnOjo6ikQiZik9vsQU3gsURbH3XiCEJCYmKmigpH7ub3vcPnKd7ebNm1OmTLGwsJDtbEuWLBGLxadOnWpqaqqsrKRHaZ07d45pgJ5GsdfTqK72lpqS2QsXLiSEyO6VKioqRCKRm5sbM8Xb29vU1FR2K3SmuPekpKQwU+jPR7KbjB4VfO3aNWZKfn4+IcTV1VXB6qi890yePFl2yvz584VCYX19Pf3wjz/+IIQsX76caZCenm5ra9va2ko/VGZL0lsjNTVVQSVyNCezlVnBpKQkQkhwcLBUKl24cOEnn3yizJx7RPcQ2f2XXA9ZtGgRIUT280FLS4uNjY2+vn5lZSVFUaGhoYSQI0eOMA3KyspEIpHsbrTHl5jCe4GiKPbeC5qT2Yq3T+fO9vjxYwMDA9nO5uTkNHHiRNmZjBw5Ujaz0dMo9noa1dXeUlOuZycnJ/P5fNnviVlZWb3yyiu5ubmlpaX0lPPnz9fW1np4ePR5KRMmTGD+tre3J4SUl5fLNjA0NBw7dizz0MXFxcbG5vr16xUVFX1eaK/MnDlT7lSqq6trW1vbrVu36IdTp051cXHZv39/TU0NPWXbtm0fffSRUCikHyqzJWl/+ctfBnBNBowyKxgaGvrpp5/++uuvnp6eNTU1W7ZsUdXS9fT0Xn/9deahXA/57bffCCEzZsxgGohEIl9f3+bmZvptf/LkSUKIv78/08DGxmbkyJGyi+jxJSZ4LxBCBv17ocft07mzWVhYjB49WvZfpk2blpmZuWzZsuzsbPqUeGFh4eTJk5kG6GlEw3qaRmS2RCKpr6+XSqVisVj24sHVq1cJIXfv3lXVgsRiMfO3rq4uIUTu8o+JiYncvwwdOpQQ8vjxY1XVoFh9ff3nn3/u4uJiampKb4SPP/6YENLU1MS0Wb16dVNT07fffksIKSoqOnv27LJly+inerUlDQ0N1bNSKqT8Cm7ZsuX111/PzMwMDQ3l81XWz+kLUbJTmB5C16anp2dsbCzbwNLSkhBSWVkpkUgaGhr09PSMjIw6z0GWgpdYVfBekF0WF98LirdPd53N1NRU9mFcXNyBAwcePHjg6+s7ZMiQadOm0Z87VQg9TXZZ/e9pfdyXye22+tlSJBKZmJgIBIK2trbOJwd8fHz6VmQf1NTUUBQlO4XuN8xelc/nt7a2yjaoq6uTm4nyG6ezgICALVu2LF26tKioSCqVUhS1c+dOQohsVe+++66lpeU333wjkUi++uqrhQsXMu9DzdmSKtF5Syq/gufPn6+vr3dxcVm+fPn169d7nLOS6uvr5aYwPUQkEonF4paWloaGBtkGVVVVhBArKyuRSGRsbNzS0tLY2CjboLa2Vm6eCl5itcF7QXN0uRkVb5/uOptcEPJ4vAULFpw+fbquri45OZmiqODg4B07dgzk2shDT+uVPma2gYEBsxFHjRoVHx/fz5bBwcHt7e1yQ3BjYmIcHBza29v7VmQftLS05OTkMA9v3LhRXl7u6upqbW1NT7G2ti4rK2MaVFZWlpSUyM1E+Y3DEAgEBQUFHR0dGRkZVlZWK1eutLCwoHthc3OzXGORSLR8+fLHjx9/9dVXBw8eXLVqleyzGrIlVaLLLanMCv75558RERH/+7//e+zYMX19/cDAwOrq6h7nrIzGxkbZTwByPWT27NmEkBMnTjANJBLJmTNn9PX16VOU06dPJ/85aUl78uRJYWGh3FIUv8TqgfeC5ui8GZXZPp07W2VlZVFRkWwbExOTgoICQohQKPTz86NHOMt2YDVAT+sd2U8Eyo+JmDZtmlgsLikpyczMFAgEt2/fpqd3HoPWXUu5MWhVVVXDhw93dnZOTU2tq6urqanZs2ePgYGB7FiP/o9glC0sKiqK/PfYB3pUsK+vr4IRjCtWrCCEfP311w0NDffu3ZszZ46tra3caIjuVlnBaAgdHZ07d+5QFPXmm28SQrZu3VpdXd3U1HT27FkHBwdCSFpammz76upqfX19Ho/XeW7KbMnOW6NHrIxB63JL9riCDQ0NY8aMOXr0KP3w/PnzQqHwjTfekB3D1d1rpBg9xtXT0zM7O7vLHiI7bvzZs2fMuPH4+Hi6wb1798zMzJihvLdu3fL396eP0eWWpeAlpvBeUG5DDdB7gah9DFqXm7HH7SPX2W7cuDFt2rRhw4bJdjaxWOzt7X39+vWWlpaqqqrNmzcTQr744gumAXoaQ/09jVLhuPGCggIvLy9DQ0N7e/u4uDiKouSugrz77rvdtdy2bZtsy08//ZRuSX+x1dnZWSgUWlhYTJ06VW6TeXl5KR7BKHepYNu2bRRFZWVldV6c7JQZM2bQ/05/jLh9+7a/v7+xsbG+vr63t3d6errsIurq6pYsWWJtba2vr+/p6ZmTk+Pm5kbPJyoqqrtV7lxbZ3Tvqa6ujoyMtLe3FwqFlpaWixYt2rBhA91AdggiRVFLly4lhFy4cKHzdlCwJeW2hvIxzEpmd7klKYUr+OGHHzKrduPGDbnD6y1btiiec3eYTmtra3v58mUfHx8jI6Mue8iTJ09Wr17t5OQkFArFYrG/v/+ZM2dkGxQWFgYFBQ0ZMoT+3svx48eZ3xuPiIiQbangJcZ7QckNNRDvBaL2zO5yMyqzfZjOZmBgMHHixAsXLkyePNnAwICZc15eXmRk5EsvvUR/P9vd3X3fvn30+WEaeppsJWruaZQKM1sryR36a7gffvhBrj8NKFYye5BT80ssC+8FBdSf2So0atQoBwcHNS9UAfQ0xTT3u17QW3v27Fm7di3bVcAAwkusJGyoLlVWVpqZmbW1tTFTiouL79+/T58Khj7QhJ6GzOaS7777bvbs2Y2NjXv27Hn69OmcOXPYrghUDC+xkrChlPH06dPIyMhHjx41NTVdvnw5LCxsyJAhmzZtYrsuLtG0nobMJuQ/v1h7/fr1srIyHo/32WefsV1Rt5KTk01NTXfv3p2QkNDbn9UFBXjdowfmqA27LzHeC1rDysqK/hLXG2+8YWpqOmvWrBEjRly+fNnZ2Znt0ghBT+srHiUzNCApKSksLIz678ECAESlfYP+ScXDhw/3f1YA6sfj8RITE/t/vIX9LfSo894Sx9kAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA1d3FaMvpEIcEtLS0tNTY2NjQ2PxxuI+ZeWlqpwbtnZ2ehmAGSA97cSiaSqqsrBwWHgFgEDKjs7293dXXbKf2W2vb19SEiIeksC1Xj8+HFOTo6+vr6Tk5Ozs7NIJFLt/O3s7FTVNzw8PFQyH+hSeXn5lStXZs2axXYhWiskJMTe3r7/8xnQ/W1tbe39+/dLS0v5fL6lpaXKdwigHu7u7nI7TB7u3qo1ysrK9u3bFxcX9+zZs8DAwGXLlk2ZMoXtokDdcFfmwUwikRw7diw2NjYjI2P06NHvv/9+RESEkZER23WByuB6tvawtbXdvHlzaWnpzz//XFZW5ufn5+bmFh8f39TUxHZpADCwHjx4sGHDBjs7u/nz59vY2KSlpd2+fXvVqlUIbC2DzNY2IpEoNDQ0IyPjypUr48ePX716tY2NTWRk5J07d9guDQBUTCqVnj59es6cOSNHjjxw4EBERMS9e/eSkpKmTJkyQENbgF3IbK3l5ua2d+/e4uLijRs3/vHHH6+++qqfn9/hw4c7OjrYLg0A+qu+vj4+Pp5+X5eXlx86dOjhw4fR0dEqudYOGguZreWGDh0aFRX14MGDP/74Q09PLywsbNSoUTExMTU1NWyXBgB9kZubGxkZaWNjs27dOi8vr/z8/PT09NDQUKFQyHZpMOCQ2YMCn8+fMmVKSkpKYWFhSEhITEyMra1teHh4Xl4e26UBgFIkEsnhw4f9/PzGjx9/4cKFzz//vKSkZO/evS4uLmyXBuqDzB5cRowYER0d/fDhw127duXl5Y0bN278+PHx8fEtLS1slwYAXSsrK9u8ebO9vf28efP09PTS0tLu3LkTFRVlYmLCdmmgbsjswcjY2HjZsmX5+flXrlx5+eWXV6xY4ejouGHDhpKSErZLA4D/w4wvc3R03Lt37+LFi//888+UlBSMLxvMkNmDmpub24EDBx4+fLhmzZqDBw86OTkFBAScPn0a3+4FYJHs+LIHDx58//33JSUlGF8GBJkNhBBra+uoqKj79+8nJCS0tLT4+fmNHj06Jiamrq6O7dIABperV69GRkba2toy48uuXLkSHh6O8WVAQ2bD/9HV1Q0NDU1LS7t69erkyZO3bNni4OAQGRl58+ZNtksD0HKtra30+DI3N7fz589v2rTp4cOHGF8GnSGzQd64ceP27t1bVla2ffv2S5cuubi4eHp6Hj58uL29ne3SALRNeXn55s2b7ezsmPFlBQUFUVFRpqambJcGmgiZDV0Ti8XLli27efNmWlqajY3NvHnzhg0btnnz5urqarZLA+A8iqLo8WXDhg2jx5c9ePAA48ugR8hsUIT+YndSUlJRUdGCBQvi4uLs7OzmzJlz+vRptksD4KTuxpfhjpmgDGQ2KMXZ2Tk6Opq+AUlpaSluQALQW9euXWPGl3l6el6/fh3jy6C3kNnQC/QNSDIzM2VvQLJq1ao///yT7dIANBQzvuy1116THV82ZswYtksD7kFmQ1/I3oDk6NGjL774Im5AAiCHHl9mb28/d+5cjC8DlUBmQ98xNyBJTk4mhISFhdFf7MYNSGCQS09Pp8eX7dmz57333sPvl4GqILOhv/h8fkBAAH0M8fbbb8fExNjZ2eEGJDAIPXv2jB5f5uXlRY8ve/ToEcaXgQohs0FlRo4cSd+AJDY2lrkByYEDB9ra2tguDWBg3blzZ9WqVfTwjtdeey0vLw/jy2AgILNBxZgbkFy6dMnZ2TkiIsLe3h43IAGtxIwve/nll3///fdNmzaVlZUdOHDA1dWV7dJAOyGzYaB4enomJSWVlJSsWbPm559/dnZ2xg1IQGuUl5fHxMQMHz587ty5hJBjx44VFhZGRUWZmZmxXRpoM2Q2DCz6BiQPHjw4dOgQfQOSl156KTY2trGxke3SAPqCGV+2c+fOd99998GDB2lpaQEBARhfBmqAzAZ1YG5Akpub6+3t/cknn9ja2kZGRt66dYvt0gCUQo8vc3FxoceXxcXFFRcXR0dHDxs2jO3SYBBBZoNavfbaa3v37i0vL9+2bdvFixdfffVV3IAENFxBQcGqVatsbW1XrVo1bty4a9euXblyZdmyZXp6emyXBoMOMhtYQN+A5NatW7gBCWgsZnzZSy+99Pvvv3/22Wf0+LKxY8eyXRoMXshsYA1zA5LCwsIFCxZ888039vb2uAEJsK6iogLjy0AzIbOBfcOHD4+Oji4rK4uPj797966fn9/48eNxAxJQP2Z82Y4dOzC+DDQQMhs0hUgkCg8Ppy8Wurm5MVcQcQMSGGhy48u++eYbjC8DzYTMBo3D3IBkw4YNycnJ9A1IUlJS8MVuULnCwsIux5fp6+uzXRpAF5DZoKEsLS2joqL+/PNP+gYkgYGBI0eOjImJqa2tZbs04DzZ8WWpqamfffZZaWkpxpeB5kNmg0ZjbkBy586dt99+Ozo62tbWNjw8/Pr162yXBpxEjy978cUX6fFlR48eLSoqioqKMjc3Z7s0gJ4hs4EbRo0aFR0dXVJSEhsbe+3atbFjx+IGJNArubm54eHh9Piyd9555/79+xhfBpyDzAYuoW9AcuPGDeYGJA4ODhs2bHj06BHbpYGGamhoiI+PHzNmzPjx42/fvs2ML3N0dGS7NIBe42FcD3BXeXl5fHz8t99+W1tbO3369FWrVvn6+g62w6aysrKAgADmfMPz58+rq6tlA2ns2LE//fQTO8WxqrCw8F//+tfevXubm5tnzZq1Zs0aDw8PtosC6BdkNnBea2vr0aNH4+PjT58+PXr06Pfffz8iIsLIyIjtutTn5ZdfvnPnTnfPbtmy5bPPPlNnPezq6OhITU3dtWvXmTNnhg8fvmTJkiVLluByNWgHnBsHzpO9Ackbb7wxCG9AEh4eLhAIuns2LCxMncWwqLKyMiYmxsnJKSgoiBCSmJhYUFCA8WWgTXCcDdqmvr5+//79sbGxxcXFvr6+y5Ytmz17toJII4RQFFVRUWFjY6O2IlWrpKTE0dGx83uZx+ONGzcuNzeXlapUoqqqaujQoT1e78jNzY2NjU1ISDA0NAwPD1+9erWTk5N6KgRQJxxng7YRi8WrVq26d+/eqVOnTE1N582b5+jouHnz5idPnnT3L2fOnJkwYcKNGzfUWacKOTg4TJgwgc+Xfzvr6OiEh4ezUpJKFBQUvP7662lpad01oMeXubq6MuPLysvLY2NjEdigtSgArXb37l367KhIJAoNDU1PT+/cJiAggBBiZGR04cIF9VeoEt98842Ojo7cu5vH45WVlbFdWh9lZWWJxWJCyMyZMzs/S9+0w9TUlH5ZMzIy1F8hgPohs2FQaG5u/vHHH11dXcl/fhu1qamJfurhw4f0ESqfzxcKhYmJieyW2jePHz+Wy2wdHZ3JkyezXVcfnTp1Sl9fn14jPp//559/0tPb29uPHTs2ZcoUHo9H31qmurqa1UoB1ArnxmFQ0NPTCw8Pz8vLu3Llyssvv7xixQobG5tVq1YVFxfv3r2bzgapVNre3j5v3rzdu3ezXW+vWVhYTJ48WS62FyxYwFY9/fHjjz9Onz5dIpF0dHQQQnR0dOLj4+nxZc7Ozsz4MvpQ+4UXXmC7XgD1wRg0GIwqKiri4+Pj4+Orqqr09PSeP38u1yAqKio6OpqV2vrsxx9/XLx4sVQqpR8KhcLHjx+bmJiwW1VvxcTEbNy4UW6/ZGho2NraKhaLIyIiIiMjcbkaBi1kNgxebW1ta9as2b17N5NzDD6fP3/+/O+//17xgHON0tDQ8MILL7S2thJCBALBW2+9dfToUbaL6oWOjo6PPvpoz549nXdKfD5/6dKl//znP/X09FipDUBD4Nw4DF5CoTArK6vLp6RS6cGDB4OCgpqbm9VcVZ8ZGxvPnDlTKBQSQjo6OubPn892Rb0gkUjmzp27d+/e7o4irly5gsAGwHE2DF5Xr151c3NT0EAgELi6up48eZIrF01//fXXkJAQiqL09fVramq4chPourq6GTNmXL58ub29XUGznJyc8ePHq60qAA2E42wYvGJjYzt/P0pWe3t7fn7+pEmTSktL1VZVf7z11lsGBgaEkLfffpsrgV1aWurh4ZGTk6M4sHV0dLg4NhBAtThzrQ40TWlpaWZmJttV9F1DQ0NCQgI9MpnB5/P5fD6Px6O/VtHR0dHW1lZUVOTi4rJp0yY7Ozu2qlXehAkTzp8/b29vn5SUxHYtPSstLd2yZUtdXR39sPP2Z1p2dHT8/PPPkyZN4vQvyc+ZM4ftEoDbcG4c+igpKWnw/JA1gEpgfwv9hONs6BfsgzRNR0fHl19+uWnTJrYLgf+Cz7igErieDaBVdHR0Nm7cyHYVADAgkNkA2oZD3ykHgF5BZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGGCxycnIWLVrk5OSkr69vZmb26quvvv3227t3775//z7bpQGAUpDZANpPKpV+/PHHEydOHDp06O+//15XV3fnzp2dO3c+e/Zs+fLlL774Ynt7O9s1AkDPkNkA/8fIyMjT01Mrl75p06bt27d/++23W7duHT16tEgksrS09PPzO3ny5PTp0wdoof2kxS8HQJ8hswG0XEFBQXR0tJub29KlS+We0tHR2bRpEytVAUAf4D67AFouPj5eKpWGhoZ2+ayHhwdFUWouCQD6BsfZMOBqamrWrl07fPhwkUhkZ2c3ZcqU/fv3Nzc3d26gq6tramo6ffr0c+fO0U8lJyfz/qO4uDgsLMzExMTc3HzmzJlyI6cUL6W9vT0xMdHPz8/KykpfX9/FxSU2NlYqldLPbt++ncfjPX/+PCMjg16WQPD/P85WV1evXLnS0dFRV1fXwsIiODg4Ly+vV+UN0NKVdPHiRULImDFjlGyPl2NAXw6AfqEA+iQxMVGZ/lNRUeHk5GRlZZWSkvLs2bPKysotW7YQQnbu3CnbwNLSMiUlpb6+vrCwMDg4mMfj7du3j5lJYGAgISQwMDAzM7OxsTEtLU1fX3/ChAnKLyUlJYUQ8uWXX9bW1lZXV+/atYvP569bt062VENDw0mTJsnVX15ePmzYMEtLyxMnTjQ0NNy8edPb21tPTy8zM1P58gZ06T4+PmZmZllZWd29BNbW1oSQf//73901kIWXo59L746S7xcAxdCHoI+U3ActWrSIEJKYmCg7cdq0aczum25w6NAh5tmWlhYbGxt9ff3Kykp6Cr0XTklJYdqEhIQQQqqrq5VcSkpKyuTJk2WfnT9/vlAorK+vZ6Z0uZteuHAhIeTgwYPMlIqKCpFI5ObmxkzpsbwBXbq3t7epqamC2KAz+/Lly901kIWXo59L7w4yG1QCfQj6SMl9kFgsJoQ8e/asVw0WLFhACPnxxx/ph/RemMkMiqLWrFlDCLl+/bqSS+ls27ZthBDZqOtyNy0Wi/l8vuzenKKo1157jRDy6NEjJcsb0KX3yM3NjRCSmpqqTGO8HP1ceneQ2aASGIMGA0gikdTX1+vp6RkbG/eqgaWlJSGksrJSdiKdBDRdXV1CCH0NsselEELq6+u/+uqr3377rbS0tK6ujpne1NTUY/1yi2bcvXvXzs6ux/LUs3QFvL29c3Nz8/Pze/xaF14OVS0dYIBgDBoMIJFIJBaLW1paGhoaetWgqqqKEGJlZaWSpRBCAgICtmzZsnTp0qKiIqlUSlHUzp07CSGUzJBpHo/Xec4mJiYCgaCtra3zB14fHx9lymN96ZGRkQKB4MiRI10+u379ej6fX1BQQPByqGXpAP2BzIaBNXv2bEJIamqq7MRx48bRpyuZBidOnGCelUgkZ86c0dfX9/f3V8lSOjo6MjIyrKysVq5caWFhQe+OZQeu0wwMDFpbW+m/R40aFR8fTwgJDg5ub2/PyMiQbRkTE+Pg4KDkb4exu3RCyMiRI//2t79duXLlhx9+kHuqsLBw7969c+bMGT16ND0FL8eALh2gv/p+Wh0Gt16NG7e2tj5+/PizZ88ePXr0wQcfWLf/vq0AACAASURBVFpaPnz4ULYBPVD52bNnzEDl+Ph4Zib0Fcrm5mZmSlRUFCHk2rVrSi7lzTffJIRs3bq1urq6qanp7NmzDg4OhJC0tDRmntOmTROLxSUlJZmZmQKB4Pbt2xRFVVVVDR8+3NnZOTU1ta6urqamZs+ePQYGBrIDrHosb0CX3uO4cdqGDRuEQmFUVFRhYaFEIiktLf3uu++sra09PT0bGxvlXi+8HH1eendwPRtUAn0I+kj5fdCTJ09Wr17t5OQkFAqtra3nzp1bVFTUXQOxWOzv73/mzBn6qaysLNmPmJ9++in13z8AMmPGDGWWUl1dHRkZaW9vLxQKLS0tFy1atGHDBnoOzKDfgoICLy8vQ0NDe3v7uLg45n/prxo7OzsLhUILC4upU6cyO3clyxugpdO8vLwUjxtnXL58ecGCBXQZxsbG7u7usbGxEolEweuFl6O3L0d3kNmgEjwKP4EEfZKUlBQWFob+A6AMvF9AJXA9GwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILMBAAC4AZkNAADADchsAAAAbkBmAwAAcAMyGwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILMBAAC4AZkNAADADchsAAAAbkBmAwAAcAMyGwAAgBsEbBcA3JaUlMR2CQAckJWVxXYJoA2Q2dAvYWFhbJcAADBY8CiKYrsGAFCZpKSksLAwvK8BtBKuZwMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHCDgO0CAKBfqqqq9u/fzzzMz88nhMTExDBTTE1Nly1bpv7CAEDleBRFsV0DAPRde3u7paVlfX29QPB/H8EpiuLxePTfEolk6dKl8fHx7BUIACqDc+MA3CYQCObOncvn8yX/0drayvxNCHnnnXfYrhEAVAPH2QCcl56e7uXl1eVTFhYWFRUVOjo6ai4JAAYCjrMBOG/SpEk2Njadp+vq6oaHhyOwAbQGMhuA83g83vz584VCodz01tbWefPmsVISAAwEnBsH0AZ5eXnjxo2Tmzhs2LDi4mI2ygGAAYHjbABtMHbs2BEjRshO0dXVXbRoEUvlAMCAQGYDaInw8HDZ0+Otra1hYWEs1gMAKodz4wBa4v79+yNGjKDf0Twez8XF5fr162wXBQCqhONsAC0xfPjwsWPH8vl8QohAIAgPD2e7IgBQMWQ2gPYIDw+nM7u9vR0nxgG0D86NA2iPiooKOzs7qVQ6ceLEjIwMtssBABXDcTaA9rC2tqZ/EG3hwoVs1wIAqofjbOC20NDQI0eOsF0FaLrExMQ5c+awXQVAf+FenMB57u7ua9asYbsKTfH8+fP4+HhsEFm4tA9aA5kNnGdnZ4dDKFl+fn52dnZsV6FBkNmgNXA9G0DbILABtBUyGwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILMBAAC4AZkNAADADchsAAAAbkBmAwAAcAMyGwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILNhMEpISODxeDweT09Pj+1a+iI1NXXkyJECgaL78inTRgEjIyOeDD6fb2pq6urqunz58tzc3L7NEwD6CZkNg9HcuXMpivL19WW7kF67f//+rFmzNm7cWFVV1Z82PWpsbLx27RohJDAwkKKotra2goKCv//97wUFBePHj3/vvfeampr6PHMA6BtkNgCXbNq0aeLEibm5ucbGxv1p01s6OjqWlpaBgYFnz55dv379/v37582bR1GUquYPAMro43kzAGDF999/r6+v3/82/REdHX3hwoVjx44lJCTMmzdv4BYEAHJwnA3AJcqE8YAGNiGEx+OtWLGCEPLtt98O6IIAQA4yGwaLgoKCoKAgsVhsaGjo5eWVnp7euU11dfXKlSsdHR11dXUtLCyCg4Pz8vLop5KTk5kBWcXFxWFhYSYmJubm5jNnzrx//z4zB4lE8vnnn48ePdrAwMDMzCwgIODYsWMdHR3KLIJDPD09CSHZ2dltbW30FGw6AHWgALgsJCQkJCSkx2Z37941MTGxtbU9depUQ0NDfn7+1KlTHR0dRSIR06a8vHzYsGGWlpYnTpxoaGi4efOmt7e3np5eZmYm0yYwMJAQEhgYmJmZ2djYmJaWpq+vP2HCBKbBkiVLxGLxqVOnmpqaKisr161bRwg5d+6c8otQkq2trY6OTp/b+Pj4mJmZZWVlKfh32TFocpqbm+kdSHl5OaXxm44QkpiY2GMzAM2HzAZuUzKzQ0NDCSFHjhxhppSVlYlEItnMXrhwISHk4MGDzJSKigqRSOTm5sZMoYMnJSVFtgBCSHV1Nf3Qyclp4sSJsoseOXIkEzzKLEJJ/cxsb29vU1NTxYGnILOZQeN0Zmv4pkNmg9bAuXEYFE6ePEkI8ff3Z6bY2NiMHDlStk1ycjKfz585cyYzxcrK6pVXXsnNzS0tLZVtOWHCBOZve3t7Qkh5eTn9cNq0aZmZmcuWLcvOzqbP6xYWFk6ePLm3ixho58+fr62t9fDw6Nu/V1RUEEKEQuELL7xABtmmA2ARMhu0n0QiaWho0NPTMzIykp0+dOhQ2Tb19fVSqVQsFsv+lsjVq1cJIXfv3pX9R7FYzPytq6tLCJFKpfTDuLi4AwcOPHjwwNfXd8iQIdOmTfvtt9/6sAgNR48G8PDwEAqF2HQAaoPMBu0nEomMjY1bWloaGxtlp9fW1sq2MTExEQgEbW1tnc9H+fj4KLksHo+3YMGC06dP19XVJScnUxQVHBy8Y8cOFS6CdVKpNC4ujhDy4YcfEmw6ADVCZsOgMH36dPKfM+S0J0+eFBYWyrYJDg5ub2/PyMiQnRgTE+Pg4NDe3q7kgkxMTAoKCgghQqHQz8+PHjJ94sQJFS6CdRs3brx8+fLs2bPpUQIEmw5AXZDZMCh8+eWXZmZmq1evTktLa2xsvH379vz58+VOlf/jH/8YPnz44sWLf//99/r6+tra2r179/7973/fvn17r361+/3338/Pz5dIJI8fP966dStFUW+++aZqF9F/b775prm5eXZ2tpLtpVLp48ePjx496uvru3Xr1sWLFx88eJDH49HPDqpNB8Cm/g1hA2CZkuPGKYoqLCwMCgoaMmQI/RWj48ePM783HhERQbepqalZu3ats7OzUCi0sLCYOnVqWloa/VRWVpbsG+fTTz+l/vuXO2fMmEFRVF5eXmRk5EsvvUR/ydjd3X3fvn1SqZQpQ8EilJGSktL5Xbxv377etvHy8lI8btzQ0FD233k8nlgsdnFx+eCDD3Jzczu31+RNRzBuHLQFj8IvBgOX0adnDx8+zHYhoLl4PF5iYuKcOXPYLgSgv3BuHAAAgBuQ2QAAANyAzAbQFLzubd68me3qAIB9GG8JoCkwuAQAFMNxNgAAADcgswEAALgBmQ0AAMANyGwAAABuQGYDAABwAzIbAACAG5DZAAAA3IDMBgAA4AZkNgAAADcgswEAALgBmQ0AAMANyGwAAABuQGYDAABwA+7rBZx35MgRHo/HdhUAAAOOh9v/AadlZWU9evSI7Sr6rqCgICYm5sUXX/zkk0808JMHRVH/+Mc/7t69+/HHH7/88stsl9N3EydOtLOzY7sKgP5CZgOw5syZM0FBQT4+PklJSXp6emyX07XW1tYFCxYcPXr04MGDb7/9NtvlAAxquJ4NwI7k5OQZM2YEBgb++uuvGhvYhBBdXd1ffvll0aJFYWFh33//PdvlAAxquJ4NwIIDBw5ERERERkbu2rWLz9f0j846Ojp79uxxcnJaunTp06dP161bx3ZFAIMUMhtA3Xbt2rV69er169dHR0ezXUsvREVFGRgYrF69+smTJ9yqHEBrILMB1ComJmbjxo3btm3761//ynYtvfbRRx+Zmpq+99579fX1cXFxmn+GAEDLILMB1ISiqLVr13799df79u2LiIhgu5w+mj9/vlgsDgsLq6ysPHTokCZfiQfQPhg3DqAOHR0dS5Ys+eWXXw4ePBgSEsJ2Of114cKFWbNmjR8/Pjk52djYmO1yAAYLZDbAgJNIJO+8887Jkyd//fVXf39/tstRjdzc3LfeemvYsGGpqakvvPAC2+UADArIbICB9fz589mzZ+fk5Bw/fnzSpElsl6NKDx488PPz09PT++OPP/CLJQBqgCEkAAPo6dOnfn5++fn5586d07LAJoQ4Ozunp6fr6Oh4eXndvXuX7XIAtB8yG2CgVFZWTp48uays7OLFi2PHjmW7nAFhbW194cIFa2trLy+vvLw8tssB0HLIbIABUVxc7OXl1dramp6ePnLkSLbLGUCmpqZpaWmurq4+Pj6XLl1iuxwAbYbMBlC9O3fueHp6Dhky5OLFi/b29myXM+AMDQ1TUlL8/Pz8/Px+++03tssB0FrIbAAVu3LlyhtvvOHs7Hz27FkLCwu2y1ETXV3dQ4cOhYeHh4aG/utf/2K7HADthN9UAVAl+ovLkyZNOnLkiIGBAdvlqJWOjs7evXttbGwiIiKePn26du1atisC0DY6mzdvZrsGAC1x/PjxoKAgPz+/I0eO6Ovrs10OC3g83uTJk01MTP761782NzdPmTKF7YoAtAqOswFUg75h5bx5877//nuBYFC/s1atWmVqahoREdHQ0PD111/jZ8kBVGVQ71kAVGX37t0rVqxYsWLFP//5Tx6Px3Y57AsPDzcxMQkLC3v69OmPP/4oFArZrghAG+DzL0B/xcTELF++/OOPP46NjUVgM2bNmvX777+fOHFi+vTpjY2NbJcDoA3w26UAfUdRVFRU1Pbt23fs2LF69Wq2y9FEubm506dPd3JySk1NNTc3Z7scAG5DZgP0UUdHxwcffPDDDz989913ixYtYrsczVVQUDB16lRjY+NTp07Z2tqyXQ4AhyGzAfqitbV1wYIFR48e/eWXX4KDg9kuR9OVlJRMnTpVIpGcOnVqxIgRbJcDwFW4ng3Qa01NTUFBQSdOnEhJSUFgK8PBwSEzM9PKyuqNN97Az5ID9BkyG6B36uvr/f39s7OzT58+7efnx3Y5nGFmZpaWlubi4uLj45Oens52OQCchMwG6IXHjx/7+Pjcu3fv/Pnz7u7ubJfDMUZGRikpKVOmTPH39//999/ZLgeAe5DZAMoqLy/39fV9+vTppUuXxowZw3Y5nCQSiRISEt55551Zs2bt37+f7XIAOAa/qQKglAcPHvj5+YlEovT0dAx+7g8dHZ34+HhTU9PFixc/ffp0zZo1bFcEwBn4vXGAnt26devNN98cOnTomTNnrKys2C6H83g8np+fn56e3vr16/Gz5ADKw3E2QA8uX7781ltvvfrqq8eOHRsyZAjb5WiPqKgoKyurJUuWNDY27tq1Cz9LDtAjZDaAImfPng0KCvL29k5KShqct+oaUAsXLhSLxfPmzXv69On+/fvxs+QAiuGDLUC3jh49OmPGjICAgF9//RWBPUCCgoJSU1NTUlJmz57d3NzMdjkAGg2ZDdC1n376KSQk5L333vvpp59w/DegfHx8zp49++9//9vHx6e2tpbtcgA0FzIboAvffPPNokWL/vrXv3777be4zqoG48ePv3jxYnl5ube3d3l5OdvlAGgo7IwA5MXExKxcuTI6Ojo6OprtWgaRl1566dKlS62trZ6envfu3WO7HABNhMyGwSsuLk4ikchOoShq7dq1n3766d69ez/++GO2Chu0hg0bdunSJRMTEy8vr/z8fLlnr169mpOTw0phABoC9/WCQSo/P3/s2LEzZ8789ddfBQIBIaSjo2PZsmU///zzTz/9NGfOHLYLHLwaGxtnz56dm5t7/PjxiRMn0hOLioo8PDxeeeWVixcvslseAItwnA2D1CeffKKjo5Oamvree+9RFNXa2hoWFpaQkHD06FEENruMjIyOHz/u4+MzderUkydPEkJKS0t9fHzq6+svXbr0xx9/sF0gAGtwnA2DUU5Ozuuvv053fj6fv3jx4pKSkn//+9/Hjx/39PRkuzogROa0R1xc3NatW4uLi9va2nR0dEaNGnXjxg0MDITBCZkNg5G3t3dmZmZ7ezv9kMfjGRsbnz9/fty4cewWBrIoilq5cuWBAweam5vb2troiTwe79ChQ2FhYezWBsAKfFaFQef06dMXL15kApsQQlHUs2fPzpw5w2JV0FlbW1tBQUFTUxMT2IQQHo+3YcMG2SkAgwcyGwadjRs30oPO5Kxfv/67775Tfz3QJalU+u67754/f1720xU9vaSk5IcffmCrMAAW4dw4DC5Hjx4NCgrq7lk+n5+UlPT222+rsyTojKKoyMjIH374oaOjo/OzPB7PwsKiuLgYPygLgw2Os2EQkUqlGzZs0NHR6fJZoVBIUdShQ4dw3pV1V65cSU5Opiiqy7FmFEXV1NR8/fXX6i8MgF3IbBhEfvnll8LCws6HbgKBQCgUhoWF3bp168iRI/h1cdZNmDChtLQ0ISGBHhXY+RXp6OjYsmULfpwcBhucG4fBoq2t7cUXXywtLZVKpfQUPp9PUZS5ufmHH3740UcfmZubs1shdCk3N3fnzp0JCQl8Pl/2FIhAIFi/fv3//M//sFgbgJohs2Gw2Lt37/Lly+nAFgqFbW1tLi4u69atmzdvHg6sNV95eXl8fPw///nPxsZGqVRK77h0dXUfPHhga2vLdnUAaoLMhkGhpaXF0dGxqqpKIBBIpdLAwMB169Yxv4sJXPH8+fOffvppx44dd+/eFQgE7e3t77///u7du9muC0BNkNkcEBoaynYJnFdUVJSfny8QCJydnYcPH25oaMh2RYp4eHisXbu2nzPZsWNHVlaWSurRQFVVVUVFRVVVVTwez9/f38jIiO2KBrXDhw+zXcJg0cW3VEHTHDlyxN3d3c7Oju1CuKq9vb20tNTV1dXJyanLb2ZrlOzsbJXMJysrKzs7293dXSVz0zSWlpaWlpYNDQ337t0rKip67bXX2K5okCotLVVVjwVlaPr+C2hr1qzBjSv6rLGx0cDAgCu/UK3C0yru7u6D4QDo2bNnxsbGPB6P7UIGo6SkJPyOrDohs0H74cSpdhsyZAjbJQCoCTeOPAAAAACZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhs4z8jIiCeDz+ebmpq6urouX748NzeX7eo4JiEhgd6Menp6bNeitdBjoc+Q2UAIIY2NjSNGjJg5c6YGzk2ZxV27do0QEhgYSFFUW1tbQUHB3//+94KCgvHjx7/33ntNTU3qqUQLzJ07l6IoX19ftgvRZuix0GfIbCCEEIqipFKpVCoduLkZGRl5enqqZP6K6ejoWFpaBgYGnj17dv369fv37583bx5FUWpYNAwctfUf5amqJPRYUB4yGwghxNjY+P79+6mpqRo4t/6Ijo5+/fXXjx07lpCQwHYtAD1DjwXFkNmgzXg83ooVKwgh3377Ldu1APQMPRYUQ2Zrg+3bt9ODWezs7HJycnx9fY2NjQ0MDHx8fDIyMug2ycnJzJiXwsLCOXPmmJub0w+/++475qmWlhZmtjU1NWvXrh0+fLiurq6pqen06dPPnTvXh7nR5T1//jwjI4OeLhAI6urqZIfhfPHFF4SQ9vZ2ZkpISIhKNg599jI7O7utrY2eUl1dvXLlSkdHR11dXQsLi+Dg4Ly8vM7rVVxcHBYWZmJiYm5uPnPmzPv37zPzlEgkn3/++ejRow0MDMzMzAICAo4dO9bR0cE0ULAIDVRQUBAUFCQWiw0NDb28vNLT0zu3UbBGynQ/Qkh7e3tiYqKfn5+VlZW+vr6Li0tsbCxzAUVBj9qwYUPn/iP3Lw8fPgwLCzM2NjY3N1+wYMHTp0+Li4sDAgKMjY2tra2XLl3a0NCg5Ooo0we67NKqejnQY0ERCjQeISQxMbHHZq6uroaGhh4eHpmZmY2NjTk5OWPGjNHV1T1//jzTJjAwkBDi7e197ty558+fZ2dn6+joVFdXM081NzfTLSsqKpycnCwtLVNSUurr6wsLC4ODg3k83r59+/owN4qiDA0NJ02aJFezv78/n8+/d++e7EQPD4+DBw8yD318fMzMzLKyshSsu+yIHjnNzc10Vy8vL6coqry8fNiwYZaWlidOnGhoaLh586a3t7eenl5mZqbcegUGBtJbMi0tTV9ff8KECUyDJUuWiMXiU6dONTU1VVZWrlu3jhBy7tw5+lllFqFASEhISEiIMi1VMp+7d++amJjY2tqeOnWqoaEhPz9/6tSpjo6OIpGIaaPMGvXY/VJSUgghX375ZW1tbXV19a5du/h8/rp162SLUdCjuuw/zL8EBwdfuXKlsbHxwIEDhJDp06cHBgZeu3atoaFhz549hJA1a9b0anV67AMKShpUPTYxMRE5ok7Y1hygfGYTQq5du8ZMyc/PJ4S4uroyU+j3dmpqaud/l0vZRYsWEUIOHTrENGhpabGxsdHX16+srOzt3KhudnB//PEHIWT58uXMlPT0dFtb29bWVmaKt7e3qamp4t2Hgj0gMwSX3gMuXLiQECL7maCiokIkErm5uckVn5KSwkyhD/rp8KAoysnJaeLEibJLGTlyJLMHVGYRCqg5s0NDQwkhR44cYaaUlZWJRCLZzFZmjXrsfikpKZMnT5Zd9Pz584VCYX19PTNFQY9SnNknTpxgprzyyiuEkAsXLjBTnJycRo0a1avV6bEPKChpUPVYZLaaYVtzQK+Os+Um2tjYMG9+6j/v7SdPnnT+d7mUFYvFhJBnz57JtlmwYAEh5Mcff+zt3Kjud3AuLi4GBgbMTAIDA6Ojo3tcWTkK9oD0GUKhUEh/DhCLxXw+XzYnKIp67bXXCCGPHj2SLZ75aEJR1Jo1awgh169fpx9+8MEHhJClS5dmZWW1t7fLLVGZRSig5sw2NjYmhDQ0NMhOdHFxkc1sZdZIme4nZ9u2bYSQzseLXfYoxZldVVXFTPHz8yOEPH/+nJni6elpbGzcq9XpsQ8oKEkZWtNjkdlqhuvZWsXExERuytChQwkhjx8/lp1oaGioeD4SiaS+vl5PT4/eoTMsLS0JIZWVlb2am2KrV69uamqiR9wUFRWdPXt22bJl/ZmhHPrqrIeHh1AopNdLKpWKxWLZq+lXr14lhNy9e1f2H+lPLTRdXV1CCHPxNS4u7sCBAw8ePPD19R0yZMi0adN+++03+qleLYJ1EomkoaFBT0/PyMhIdjrdbZg2Sq6R4u5XX1//+eefu7i4mJqa0nP4+OOPCSGdv4vchx41ZMgQ5m8+n6+jo2NgYMBM0dHRYV47VfWBgYMeCwogs7VKTU0N9d9f66R3l7K7YGWIRCKxWNzS0iI3cqeqqooQYmVl1YfaeDxel9PfffddS0vLb775RiKRfPXVVwsXLjQ1Ne3D/LsklUrj4uIIIR9++CEhRCQSmZiYCASCtra2zh9gfXx8lF+XBQsWnD59uq6uLjk5maKo4ODgHTt2qHAR6iESiYyNjVtaWhobG2Wn19bWyrZRco0Ud7+AgIAtW7YsXbq0qKhIKpVSFLVz505CCKXcF5G76z+9pcIXSFUlyUKPBcWQ2VqlpaUlJyeHeXjjxo3y8nJXV1dra+vezmr27NmEkBMnTjBTJBLJmTNn9PX1/f39+1CbgYFBa2sr/feoUaPi4+Ppv0Ui0fLlyx8/fvzVV18dPHhw1apVfZh5dzZu3Hj58uXZs2fTV20JIcHBwe3t7bLjmQkhMTExDg4O7e3tSs7WxMSkoKCAECIUCv38/Oixu8y2Uski1Gb69OmEkJMnTzJTnjx5UlhYKNtGyTVS0P06OjoyMjKsrKxWrlxpYWFBpx0z2EoZ3fWfPlDVC6TCkhjosdADRSfOQTMQpa9ni8ViX1/fHseNy15m7u4p2XHjz549Y8aNx8fH92FuFEVNmzZNLBaXlJRkZmYKBILbt28zT1VXV+vr6/N4vC4v7/V2FG5HR0dVVVVycvKbb75JCFm8eHFTUxPTsqqqavjw4c7OzqmpqXV1dTU1NXv27DEwMJDdwp2Lj4qKIjIDrMRisbe39/Xr11taWqqqqjZv3kwI+eKLL5RfhAJqvp597949MzMzZtz4rVu3/P39hw4dKns9W5k16rH70S/H1q1bq6urm5qazp496+DgQAhJS0tjZqKgR3XXfzr/i7+/v46Ojuz/ent7y15rV0kfUFDSoOqxuJ6tZtjWHKB8Ztva2t6+fdvf39/Y2FhfX9/b2zs9PZ1+Nisrq7uPa8yVLdq7775LT3/y5Mnq1audnJyEQqFYLPb39z9z5kyf51ZQUODl5WVoaGhvbx8XFydX/NKlS8l/j/VleHl5KR6FK3f5k8fjicViFxeXDz74IDc3t3N7+nvnzs7OQqHQwsJi6tSpTGzIrdenn35K/feZ2xkzZlAUlZeXFxkZ+dJLL9HfdnV3d9+3bx99vrfHRfRIzZlNUVRhYWFQUNCQIUPoLwgdP36c+b3xiIgIuk2Pa6S4+1EUVV1dHRkZaW9vLxQKLS0tFy1atGHDBnopbm5uCnoUrXP/6fxiyR7lE0L+8Y9/XLp0SXbK3/72tx5XR8k+0GVJtEHVY5HZasaj8Ku2Go/H4yUmJs6ZM0dxs7Fjxz558qS0tFQ9VanWv/71r7i4uCtXrrBdCPvok6KHDx/WkPkoidPdD/osKSkpLCwMOaI2uJ4NGmHPnj1r165luwoAXiXmTAAAAS9JREFUAI2GzAbWfPfdd7Nnz25sbNyzZ8/Tp097PJEAADDIIbO1Af3rx9evXy8rK+PxeJ999hnbFSkrOTnZ1NR09+7dCQkJKvzFZlAn7nY/AM7BXlIbrFu3jv4BYW5ZsmTJkiVL2K4C+ouj3Q+Ai3CcDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgPt6ccPOnTsPHz7MdhWgDtnZ2e7u7qqaVWhoqEpmBdCl0tJStksYXJDZHBASEsJ2CaA+7u7uHh4e/Z+PSmYCoJidnR12UOrEoyiK7RoAAACgZ7ieDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAb/h8iPxs/N92NoQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 385 ms (started: 2021-07-27 16:18:14 +08:00)\n"
     ]
    }
   ],
   "source": [
    "plot_model(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "incomplete-virginia",
   "metadata": {},
   "source": [
    "函数式 API 是一种简单的、类似于乐高积木的方法，但非常灵活，可以定义此类任意层图。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "attractive-copper",
   "metadata": {},
   "source": [
    "**训练多输入多输出模型** "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "settled-diagram",
   "metadata": {},
   "source": [
    "您可以用与训练序列模型大致相同的方式训练模型，方法是使用输入和输出数据列表调用 fit()。 这些数据列表应遵循与传递给 Model() 构造函数的顺序相同的顺序。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "cheap-flooring",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "40/40 [==============================] - 1s 13ms/step - loss: 14.2303 - priority_loss: 0.3307 - department_loss: 13.8996 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2625\n",
      "Epoch 2/10\n",
      "40/40 [==============================] - 1s 13ms/step - loss: 7.4399 - priority_loss: 0.3307 - department_loss: 7.1092 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2422\n",
      "Epoch 3/10\n",
      "40/40 [==============================] - 1s 14ms/step - loss: 12.3965 - priority_loss: 0.3307 - department_loss: 12.0658 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2453\n",
      "Epoch 4/10\n",
      "40/40 [==============================] - 1s 14ms/step - loss: 30.4116 - priority_loss: 0.3307 - department_loss: 30.0810 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2508\n",
      "Epoch 5/10\n",
      "40/40 [==============================] - 1s 14ms/step - loss: 31.4993 - priority_loss: 0.3307 - department_loss: 31.1687 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2328\n",
      "Epoch 6/10\n",
      "40/40 [==============================] - 1s 14ms/step - loss: 36.5177 - priority_loss: 0.3307 - department_loss: 36.1871 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2555\n",
      "Epoch 7/10\n",
      "40/40 [==============================] - 1s 15ms/step - loss: 34.9692 - priority_loss: 0.3307 - department_loss: 34.6386 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2477\n",
      "Epoch 8/10\n",
      "40/40 [==============================] - 1s 14ms/step - loss: 41.7089 - priority_loss: 0.3307 - department_loss: 41.3782 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2766\n",
      "Epoch 9/10\n",
      "40/40 [==============================] - 1s 13ms/step - loss: 66.8339 - priority_loss: 0.3307 - department_loss: 66.5033 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2555\n",
      "Epoch 10/10\n",
      "40/40 [==============================] - 1s 14ms/step - loss: 84.1835 - priority_loss: 0.3307 - department_loss: 83.8529 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2664\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fc3f01caeb0>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 6.89 s (started: 2021-07-27 16:23:27 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "num_samples = 1280\n",
    "\n",
    "title_data = np.random.randint(0, 2, size=(num_samples, vocabulary_size))\n",
    "text_body_data = np.random.randint(0, 2, size=(num_samples, vocabulary_size)) \n",
    "tags_data = np.random.randint(0, 2, size=(num_samples, num_tags))\n",
    "\n",
    "priority_data = np.random.random(size=(num_samples, 1))\n",
    "department_data = np.random.randint(0, 2, size=(num_samples, num_departments)) \n",
    "\n",
    "model.compile(optimizer=\"adam\",\n",
    "              loss=[\"mean_squared_error\", \"categorical_crossentropy\"],\n",
    "              metrics=[[\"mean_absolute_error\"], [\"accuracy\"]])\n",
    "\n",
    "model.fit([title_data, text_body_data, tags_data],\n",
    "          [priority_data, department_data],\n",
    "          epochs=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "configured-trial",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "40/40 [==============================] - 1s 12ms/step - loss: 60.3639 - priority_loss: 0.3307 - department_loss: 60.0333 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2453\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[60.36393356323242,\n",
       " 0.3306571841239929,\n",
       " 60.03327560424805,\n",
       " 0.50081866979599,\n",
       " 0.24531249701976776]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 842 ms (started: 2021-07-27 16:23:56 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.evaluate([title_data, text_body_data, tags_data],\n",
    "               [priority_data, department_data])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "vocational-delicious",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 451 ms (started: 2021-07-27 16:24:05 +08:00)\n"
     ]
    }
   ],
   "source": [
    "priority_preds, department_preds = model.predict([title_data, text_body_data, tags_data])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prime-accountability",
   "metadata": {},
   "source": [
    "如果您不想依赖输入顺序（例如因为您有许多输入或输出），您还可以利用您为对象和输出层提供的名称，并通过字典传递输入数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "raised-thriller",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "40/40 [==============================] - 1s 13ms/step - loss: 92.4883 - priority_loss: 0.3307 - department_loss: 92.1576 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.2508\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fc3d6823e80>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.49 s (started: 2021-07-27 16:27:12 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer=\"adam\", \n",
    "              loss={\"priority\": \"mean_squared_error\", \"department\": \"categorical_crossentropy\"},\n",
    "              metrics={\"priority\": [\"mean_absolute_error\"], \"department\": [\"accuracy\"]})\n",
    "model.fit({\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data},\n",
    "          {\"priority\": priority_data, \"department\": department_data},\n",
    "          epochs=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "north-witness",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "40/40 [==============================] - ETA: 0s - loss: 47.3248 - priority_loss: 0.3288 - department_loss: 46.9961 - priority_mean_absolute_error: 0.4988 - department_accuracy: 0.06 - 1s 13ms/step - loss: 47.3725 - priority_loss: 0.3307 - department_loss: 47.0418 - priority_mean_absolute_error: 0.5008 - department_accuracy: 0.0648\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[47.372493743896484,\n",
       " 0.3306571841239929,\n",
       " 47.04183578491211,\n",
       " 0.50081866979599,\n",
       " 0.06484375149011612]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.05 s (started: 2021-07-27 16:27:22 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.evaluate({\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data},\n",
    "               {\"priority\": priority_data, \"department\": department_data})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "placed-opposition",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 466 ms (started: 2021-07-27 16:27:32 +08:00)\n"
     ]
    }
   ],
   "source": [
    "priority_preds, department_preds = model.predict(\n",
    "    {\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prescribed-accuracy",
   "metadata": {},
   "source": [
    "**函数API的威力：访问层连接**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "blocked-indonesia",
   "metadata": {},
   "source": [
    "功能模型是一种显式的图形数据结构。 这使得检查层如何连接成为可能，重用以前的图节点和（层输出）作为新模型的一部分。 它也非常符合大多数研究人员在考虑深度神经网络时使用的“心理模型”：层图。\n",
    "\n",
    "这实现了两个重要的用例：模型可视化和特征提取。 让我们来看看。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "revised-spice",
   "metadata": {},
   "source": [
    "> 绘制层连接性"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sonic-eleven",
   "metadata": {},
   "source": [
    "让我们可视化我们刚刚定义的模型（模型的）的连通性。 您可以使用 plot_model() 实用程序将功能模型绘制为图形："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "informational-driving",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApEAAAFgCAIAAABsb7W9AAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVwTd/4/8E9CQjgNR5EbAevRA9GiW1AoUkS0iiAF0VbRikprrdda0bZ2faz9dkGtLrZUxbZrba2AflsUpVa85VoRRbwAjyJyiiAIAuHI/P6Y784vGyAECJlMeD3/IpMPM++ZfDKvzMwnGR5FUQQAAAA0Hp/tAgAAAEApyGwAAABuQGYDAABwAzIbAACAGwSyD7Kysnbs2MFWKaD5Dh8+3P+Z7NixIysrq//zAWDL2rVrPTw8+jkT7G9BGR4eHmvXrmUe/tdx9qNHj44cOaL2koADSktLVdU3srKysrOzVTIrAPU7cuTIo0eP+j8f7G+hR9nZ2XJHOILOjVRyLAVaJikpKSwsTFVzc3d3RzcDjuLxeCqcG94IoEBoaKjcFFzPBgAA4AZkNgAAADcgswEAALgBmQ0AAMANyGwAAABuQGYDAABwAzIbAACAG5DZAAAA3IDMBgAA4AZkNgAAADcgswEAALgBmQ0AAMANyGwAAABuYCGzt2/fzuPxeDyenZ2daudsZGTEk7F9+3bVzr8/NLk26NEAddqEhAR6tnp6eiqcLdHs/qbJtUFvafKrqcm19ZkqM7uxsXHEiBEzZ85U3GzdunUURbm6uqpw0UwB165dI4QEBgZSFLVu3TqVL6LPNLk26NEAddq5c+dSFOXr66va2RLN7m+aXBv0lia/mppcW5/1MbONjIw8PT3lJlIUJZVKpVJpjy212yBc5QEycFsSr5F6YDurBDZjjwbPJhKocF7Gxsb3799X4QwBAACAgTFoAAAA3NDrzKYH4zx//jwjI4O+sC8QCAghycnJzKX+lpYWBS27U11dvXLlSkdHR11dXQsLi+Dg4Ly8vD6vGEO2sOLi4rCwMBMTE3Nz85kzZzJnBWRHGOXk5Pj6+hobGxsYGPj4+GRkZNBtvvjiC7oNcwbm5MmT9JQXXnhB8cZRXnt7e2Jiop+fn5WVlb6+vouLS2xsLH25oa6uTnY8xRdffEG3Z6aEhITQM1GwJWW3RmFh4Zw5c8zNzemHT5486deGVinFW1LBCnp6ejIrOH/+fELIlClTmCl1dXX9f40KCgpmzJghFovlegitpqZm7dq1w4cP19XVNTU1nT59+rlz5+T+PSgoSCwWGxoaenl5paenM08p/xL3Dd4LnHsvKNiMCrYPg+lsBgYGf/nLX44fP868HZYsWUIIkUgkn3/++ejRow0MDMzMzAICAo4dO9bR0dHPstHTBrCnUTISExPlpnTH0NBw0qRJnacHBgYSQpqbm3ts6erqamtryzwsLy8fNmyYpaXliRMnGhoabt686e3traenl5mZybTx8fExMzPLyspSUJjsiIPOhQUGBmZmZjY2Nqalpenr60+YMEGuJENDQw8PD7pNTk7OmDFjdHV1z58/r2B13NzczM3Nldk43dUmKyUlhRDy5Zdf1tbWVldX79q1i8/n0wOgaP7+/nw+/969e7L/5eHhcfDgQfpvZbYkvTW8vb3PnTv3/Pnz7OxsHR2d6upqBYUp3zd6FBISEhISokzLLrdkjyuYl5dnaGjo6ura2NhIUVRLS8vrr79+6NChHufcI1dXV7FY7OPjk56e3tDQ0LmHVFRUODk5WVpapqSk1NfXFxYWBgcH83i8ffv20Q3u3r1rYmJia2t76tSphoaG/Pz8qVOnOjo6ikQiZik9vsQU3gsURbH3XiCEJCYmKmigpH7ub3vcPnKd7ebNm1OmTLGwsJDtbEuWLBGLxadOnWpqaqqsrKRHaZ07d45pgJ5GsdfTqK72lpqS2QsXLiSEyO6VKioqRCKRm5sbM8Xb29vU1FR2K3SmuPekpKQwU+jPR7KbjB4VfO3aNWZKfn4+IcTV1VXB6qi890yePFl2yvz584VCYX19Pf3wjz/+IIQsX76caZCenm5ra9va2ko/VGZL0lsjNTVVQSVyNCezlVnBpKQkQkhwcLBUKl24cOEnn3yizJx7RPcQ2f2XXA9ZtGgRIUT280FLS4uNjY2+vn5lZSVFUaGhoYSQI0eOMA3KyspEIpHsbrTHl5jCe4GiKPbeC5qT2Yq3T+fO9vjxYwMDA9nO5uTkNHHiRNmZjBw5Ujaz0dMo9noa1dXeUlOuZycnJ/P5fNnviVlZWb3yyiu5ubmlpaX0lPPnz9fW1np4ePR5KRMmTGD+tre3J4SUl5fLNjA0NBw7dizz0MXFxcbG5vr16xUVFX1eaK/MnDlT7lSqq6trW1vbrVu36IdTp051cXHZv39/TU0NPWXbtm0fffSRUCikHyqzJWl/+ctfBnBNBowyKxgaGvrpp5/++uuvnp6eNTU1W7ZsUdXS9fT0Xn/9deahXA/57bffCCEzZsxgGohEIl9f3+bmZvptf/LkSUKIv78/08DGxmbkyJGyi+jxJSZ4LxBCBv17ocft07mzWVhYjB49WvZfpk2blpmZuWzZsuzsbPqUeGFh4eTJk5kG6GlEw3qaRmS2RCKpr6+XSqVisVj24sHVq1cJIXfv3lXVgsRiMfO3rq4uIUTu8o+JiYncvwwdOpQQ8vjxY1XVoFh9ff3nn3/u4uJiampKb4SPP/6YENLU1MS0Wb16dVNT07fffksIKSoqOnv27LJly+inerUlDQ0N1bNSKqT8Cm7ZsuX111/PzMwMDQ3l81XWz+kLUbJTmB5C16anp2dsbCzbwNLSkhBSWVkpkUgaGhr09PSMjIw6z0GWgpdYVfBekF0WF98LirdPd53N1NRU9mFcXNyBAwcePHjg6+s7ZMiQadOm0Z87VQg9TXZZ/e9pfdyXye22+tlSJBKZmJgIBIK2trbOJwd8fHz6VmQf1NTUUBQlO4XuN8xelc/nt7a2yjaoq6uTm4nyG6ezgICALVu2LF26tKioSCqVUhS1c+dOQohsVe+++66lpeU333wjkUi++uqrhQsXMu9DzdmSKtF5Syq/gufPn6+vr3dxcVm+fPn169d7nLOS6uvr5aYwPUQkEonF4paWloaGBtkGVVVVhBArKyuRSGRsbNzS0tLY2CjboLa2Vm6eCl5itcF7QXN0uRkVb5/uOptcEPJ4vAULFpw+fbquri45OZmiqODg4B07dgzk2shDT+uVPma2gYEBsxFHjRoVHx/fz5bBwcHt7e1yQ3BjYmIcHBza29v7VmQftLS05OTkMA9v3LhRXl7u6upqbW1NT7G2ti4rK2MaVFZWlpSUyM1E+Y3DEAgEBQUFHR0dGRkZVlZWK1eutLCwoHthc3OzXGORSLR8+fLHjx9/9dVXBw8eXLVqleyzGrIlVaLLLanMCv75558RERH/+7//e+zYMX19/cDAwOrq6h7nrIzGxkbZTwByPWT27NmEkBMnTjANJBLJmTNn9PX16VOU06dPJ/85aUl78uRJYWGh3FIUv8TqgfeC5ui8GZXZPp07W2VlZVFRkWwbExOTgoICQohQKPTz86NHOMt2YDVAT+sd2U8Eyo+JmDZtmlgsLikpyczMFAgEt2/fpqd3HoPWXUu5MWhVVVXDhw93dnZOTU2tq6urqanZs2ePgYGB7FiP/o9glC0sKiqK/PfYB3pUsK+vr4IRjCtWrCCEfP311w0NDffu3ZszZ46tra3caIjuVlnBaAgdHZ07d+5QFPXmm28SQrZu3VpdXd3U1HT27FkHBwdCSFpammz76upqfX19Ho/XeW7KbMnOW6NHrIxB63JL9riCDQ0NY8aMOXr0KP3w/PnzQqHwjTfekB3D1d1rpBg9xtXT0zM7O7vLHiI7bvzZs2fMuPH4+Hi6wb1798zMzJihvLdu3fL396eP0eWWpeAlpvBeUG5DDdB7gah9DFqXm7HH7SPX2W7cuDFt2rRhw4bJdjaxWOzt7X39+vWWlpaqqqrNmzcTQr744gumAXoaQ/09jVLhuPGCggIvLy9DQ0N7e/u4uDiKouSugrz77rvdtdy2bZtsy08//ZRuSX+x1dnZWSgUWlhYTJ06VW6TeXl5KR7BKHepYNu2bRRFZWVldV6c7JQZM2bQ/05/jLh9+7a/v7+xsbG+vr63t3d6errsIurq6pYsWWJtba2vr+/p6ZmTk+Pm5kbPJyoqqrtV7lxbZ3Tvqa6ujoyMtLe3FwqFlpaWixYt2rBhA91AdggiRVFLly4lhFy4cKHzdlCwJeW2hvIxzEpmd7klKYUr+OGHHzKrduPGDbnD6y1btiiec3eYTmtra3v58mUfHx8jI6Mue8iTJ09Wr17t5OQkFArFYrG/v/+ZM2dkGxQWFgYFBQ0ZMoT+3svx48eZ3xuPiIiQbangJcZ7QckNNRDvBaL2zO5yMyqzfZjOZmBgMHHixAsXLkyePNnAwICZc15eXmRk5EsvvUR/P9vd3X3fvn30+WEaeppsJWruaZQKM1sryR36a7gffvhBrj8NKFYye5BT80ssC+8FBdSf2So0atQoBwcHNS9UAfQ0xTT3u17QW3v27Fm7di3bVcAAwkusJGyoLlVWVpqZmbW1tTFTiouL79+/T58Khj7QhJ6GzOaS7777bvbs2Y2NjXv27Hn69OmcOXPYrghUDC+xkrChlPH06dPIyMhHjx41NTVdvnw5LCxsyJAhmzZtYrsuLtG0nobMJuQ/v1h7/fr1srIyHo/32WefsV1Rt5KTk01NTXfv3p2QkNDbn9UFBXjdowfmqA27LzHeC1rDysqK/hLXG2+8YWpqOmvWrBEjRly+fNnZ2Znt0ghBT+srHiUzNCApKSksLIz678ECAESlfYP+ScXDhw/3f1YA6sfj8RITE/t/vIX9LfSo894Sx9kAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA1d3FaMvpEIcEtLS0tNTY2NjQ2PxxuI+ZeWlqpwbtnZ2ehmAGSA97cSiaSqqsrBwWHgFgEDKjs7293dXXbKf2W2vb19SEiIeksC1Xj8+HFOTo6+vr6Tk5Ozs7NIJFLt/O3s7FTVNzw8PFQyH+hSeXn5lStXZs2axXYhWiskJMTe3r7/8xnQ/W1tbe39+/dLS0v5fL6lpaXKdwigHu7u7nI7TB7u3qo1ysrK9u3bFxcX9+zZs8DAwGXLlk2ZMoXtokDdcFfmwUwikRw7diw2NjYjI2P06NHvv/9+RESEkZER23WByuB6tvawtbXdvHlzaWnpzz//XFZW5ufn5+bmFh8f39TUxHZpADCwHjx4sGHDBjs7u/nz59vY2KSlpd2+fXvVqlUIbC2DzNY2IpEoNDQ0IyPjypUr48ePX716tY2NTWRk5J07d9guDQBUTCqVnj59es6cOSNHjjxw4EBERMS9e/eSkpKmTJkyQENbgF3IbK3l5ua2d+/e4uLijRs3/vHHH6+++qqfn9/hw4c7OjrYLg0A+qu+vj4+Pp5+X5eXlx86dOjhw4fR0dEqudYOGguZreWGDh0aFRX14MGDP/74Q09PLywsbNSoUTExMTU1NWyXBgB9kZubGxkZaWNjs27dOi8vr/z8/PT09NDQUKFQyHZpMOCQ2YMCn8+fMmVKSkpKYWFhSEhITEyMra1teHh4Xl4e26UBgFIkEsnhw4f9/PzGjx9/4cKFzz//vKSkZO/evS4uLmyXBuqDzB5cRowYER0d/fDhw127duXl5Y0bN278+PHx8fEtLS1slwYAXSsrK9u8ebO9vf28efP09PTS0tLu3LkTFRVlYmLCdmmgbsjswcjY2HjZsmX5+flXrlx5+eWXV6xY4ejouGHDhpKSErZLA4D/w4wvc3R03Lt37+LFi//888+UlBSMLxvMkNmDmpub24EDBx4+fLhmzZqDBw86OTkFBAScPn0a3+4FYJHs+LIHDx58//33JSUlGF8GBJkNhBBra+uoqKj79+8nJCS0tLT4+fmNHj06Jiamrq6O7dIABperV69GRkba2toy48uuXLkSHh6O8WVAQ2bD/9HV1Q0NDU1LS7t69erkyZO3bNni4OAQGRl58+ZNtksD0HKtra30+DI3N7fz589v2rTp4cOHGF8GnSGzQd64ceP27t1bVla2ffv2S5cuubi4eHp6Hj58uL29ne3SALRNeXn55s2b7ezsmPFlBQUFUVFRpqambJcGmgiZDV0Ti8XLli27efNmWlqajY3NvHnzhg0btnnz5urqarZLA+A8iqLo8WXDhg2jx5c9ePAA48ugR8hsUIT+YndSUlJRUdGCBQvi4uLs7OzmzJlz+vRptksD4KTuxpfhjpmgDGQ2KMXZ2Tk6Opq+AUlpaSluQALQW9euXWPGl3l6el6/fh3jy6C3kNnQC/QNSDIzM2VvQLJq1ao///yT7dIANBQzvuy1116THV82ZswYtksD7kFmQ1/I3oDk6NGjL774Im5AAiCHHl9mb28/d+5cjC8DlUBmQ98xNyBJTk4mhISFhdFf7MYNSGCQS09Pp8eX7dmz57333sPvl4GqILOhv/h8fkBAAH0M8fbbb8fExNjZ2eEGJDAIPXv2jB5f5uXlRY8ve/ToEcaXgQohs0FlRo4cSd+AJDY2lrkByYEDB9ra2tguDWBg3blzZ9WqVfTwjtdeey0vLw/jy2AgILNBxZgbkFy6dMnZ2TkiIsLe3h43IAGtxIwve/nll3///fdNmzaVlZUdOHDA1dWV7dJAOyGzYaB4enomJSWVlJSsWbPm559/dnZ2xg1IQGuUl5fHxMQMHz587ty5hJBjx44VFhZGRUWZmZmxXRpoM2Q2DCz6BiQPHjw4dOgQfQOSl156KTY2trGxke3SAPqCGV+2c+fOd99998GDB2lpaQEBARhfBmqAzAZ1YG5Akpub6+3t/cknn9ja2kZGRt66dYvt0gCUQo8vc3FxoceXxcXFFRcXR0dHDxs2jO3SYBBBZoNavfbaa3v37i0vL9+2bdvFixdfffVV3IAENFxBQcGqVatsbW1XrVo1bty4a9euXblyZdmyZXp6emyXBoMOMhtYQN+A5NatW7gBCWgsZnzZSy+99Pvvv3/22Wf0+LKxY8eyXRoMXshsYA1zA5LCwsIFCxZ888039vb2uAEJsK6iogLjy0AzIbOBfcOHD4+Oji4rK4uPj797966fn9/48eNxAxJQP2Z82Y4dOzC+DDQQMhs0hUgkCg8Ppy8Wurm5MVcQcQMSGGhy48u++eYbjC8DzYTMBo3D3IBkw4YNycnJ9A1IUlJS8MVuULnCwsIux5fp6+uzXRpAF5DZoKEsLS2joqL+/PNP+gYkgYGBI0eOjImJqa2tZbs04DzZ8WWpqamfffZZaWkpxpeB5kNmg0ZjbkBy586dt99+Ozo62tbWNjw8/Pr162yXBpxEjy978cUX6fFlR48eLSoqioqKMjc3Z7s0gJ4hs4EbRo0aFR0dXVJSEhsbe+3atbFjx+IGJNArubm54eHh9Piyd9555/79+xhfBpyDzAYuoW9AcuPGDeYGJA4ODhs2bHj06BHbpYGGamhoiI+PHzNmzPjx42/fvs2ML3N0dGS7NIBe42FcD3BXeXl5fHz8t99+W1tbO3369FWrVvn6+g62w6aysrKAgADmfMPz58+rq6tlA2ns2LE//fQTO8WxqrCw8F//+tfevXubm5tnzZq1Zs0aDw8PtosC6BdkNnBea2vr0aNH4+PjT58+PXr06Pfffz8iIsLIyIjtutTn5ZdfvnPnTnfPbtmy5bPPPlNnPezq6OhITU3dtWvXmTNnhg8fvmTJkiVLluByNWgHnBsHzpO9Ackbb7wxCG9AEh4eLhAIuns2LCxMncWwqLKyMiYmxsnJKSgoiBCSmJhYUFCA8WWgTXCcDdqmvr5+//79sbGxxcXFvr6+y5Ytmz17toJII4RQFFVRUWFjY6O2IlWrpKTE0dGx83uZx+ONGzcuNzeXlapUoqqqaujQoT1e78jNzY2NjU1ISDA0NAwPD1+9erWTk5N6KgRQJxxng7YRi8WrVq26d+/eqVOnTE1N582b5+jouHnz5idPnnT3L2fOnJkwYcKNGzfUWacKOTg4TJgwgc+Xfzvr6OiEh4ezUpJKFBQUvP7662lpad01oMeXubq6MuPLysvLY2NjEdigtSgArXb37l367KhIJAoNDU1PT+/cJiAggBBiZGR04cIF9VeoEt98842Ojo7cu5vH45WVlbFdWh9lZWWJxWJCyMyZMzs/S9+0w9TUlH5ZMzIy1F8hgPohs2FQaG5u/vHHH11dXcl/fhu1qamJfurhw4f0ESqfzxcKhYmJieyW2jePHz+Wy2wdHZ3JkyezXVcfnTp1Sl9fn14jPp//559/0tPb29uPHTs2ZcoUHo9H31qmurqa1UoB1ArnxmFQ0NPTCw8Pz8vLu3Llyssvv7xixQobG5tVq1YVFxfv3r2bzgapVNre3j5v3rzdu3ezXW+vWVhYTJ48WS62FyxYwFY9/fHjjz9Onz5dIpF0dHQQQnR0dOLj4+nxZc7Ozsz4MvpQ+4UXXmC7XgD1wRg0GIwqKiri4+Pj4+Orqqr09PSeP38u1yAqKio6OpqV2vrsxx9/XLx4sVQqpR8KhcLHjx+bmJiwW1VvxcTEbNy4UW6/ZGho2NraKhaLIyIiIiMjcbkaBi1kNgxebW1ta9as2b17N5NzDD6fP3/+/O+//17xgHON0tDQ8MILL7S2thJCBALBW2+9dfToUbaL6oWOjo6PPvpoz549nXdKfD5/6dKl//znP/X09FipDUBD4Nw4DF5CoTArK6vLp6RS6cGDB4OCgpqbm9VcVZ8ZGxvPnDlTKBQSQjo6OubPn892Rb0gkUjmzp27d+/e7o4irly5gsAGwHE2DF5Xr151c3NT0EAgELi6up48eZIrF01//fXXkJAQiqL09fVramq4chPourq6GTNmXL58ub29XUGznJyc8ePHq60qAA2E42wYvGJjYzt/P0pWe3t7fn7+pEmTSktL1VZVf7z11lsGBgaEkLfffpsrgV1aWurh4ZGTk6M4sHV0dLg4NhBAtThzrQ40TWlpaWZmJttV9F1DQ0NCQgI9MpnB5/P5fD6Px6O/VtHR0dHW1lZUVOTi4rJp0yY7Ozu2qlXehAkTzp8/b29vn5SUxHYtPSstLd2yZUtdXR39sPP2Z1p2dHT8/PPPkyZN4vQvyc+ZM4ftEoDbcG4c+igpKWnw/JA1gEpgfwv9hONs6BfsgzRNR0fHl19+uWnTJrYLgf+Cz7igErieDaBVdHR0Nm7cyHYVADAgkNkA2oZD3ykHgF5BZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGGCxycnIWLVrk5OSkr69vZmb26quvvv3227t3775//z7bpQGAUpDZANpPKpV+/PHHEydOHDp06O+//15XV3fnzp2dO3c+e/Zs+fLlL774Ynt7O9s1AkDPkNkA/8fIyMjT01Mrl75p06bt27d/++23W7duHT16tEgksrS09PPzO3ny5PTp0wdoof2kxS8HQJ8hswG0XEFBQXR0tJub29KlS+We0tHR2bRpEytVAUAf4D67AFouPj5eKpWGhoZ2+ayHhwdFUWouCQD6BsfZMOBqamrWrl07fPhwkUhkZ2c3ZcqU/fv3Nzc3d26gq6tramo6ffr0c+fO0U8lJyfz/qO4uDgsLMzExMTc3HzmzJlyI6cUL6W9vT0xMdHPz8/KykpfX9/FxSU2NlYqldLPbt++ncfjPX/+PCMjg16WQPD/P85WV1evXLnS0dFRV1fXwsIiODg4Ly+vV+UN0NKVdPHiRULImDFjlGyPl2NAXw6AfqEA+iQxMVGZ/lNRUeHk5GRlZZWSkvLs2bPKysotW7YQQnbu3CnbwNLSMiUlpb6+vrCwMDg4mMfj7du3j5lJYGAgISQwMDAzM7OxsTEtLU1fX3/ChAnKLyUlJYUQ8uWXX9bW1lZXV+/atYvP569bt062VENDw0mTJsnVX15ePmzYMEtLyxMnTjQ0NNy8edPb21tPTy8zM1P58gZ06T4+PmZmZllZWd29BNbW1oSQf//73901kIWXo59L746S7xcAxdCHoI+U3ActWrSIEJKYmCg7cdq0aczum25w6NAh5tmWlhYbGxt9ff3Kykp6Cr0XTklJYdqEhIQQQqqrq5VcSkpKyuTJk2WfnT9/vlAorK+vZ6Z0uZteuHAhIeTgwYPMlIqKCpFI5ObmxkzpsbwBXbq3t7epqamC2KAz+/Lly901kIWXo59L7w4yG1QCfQj6SMl9kFgsJoQ8e/asVw0WLFhACPnxxx/ph/RemMkMiqLWrFlDCLl+/bqSS+ls27ZthBDZqOtyNy0Wi/l8vuzenKKo1157jRDy6NEjJcsb0KX3yM3NjRCSmpqqTGO8HP1ceneQ2aASGIMGA0gikdTX1+vp6RkbG/eqgaWlJSGksrJSdiKdBDRdXV1CCH0NsselEELq6+u/+uqr3377rbS0tK6ujpne1NTUY/1yi2bcvXvXzs6ux/LUs3QFvL29c3Nz8/Pze/xaF14OVS0dYIBgDBoMIJFIJBaLW1paGhoaetWgqqqKEGJlZaWSpRBCAgICtmzZsnTp0qKiIqlUSlHUzp07CSGUzJBpHo/Xec4mJiYCgaCtra3zB14fHx9lymN96ZGRkQKB4MiRI10+u379ej6fX1BQQPByqGXpAP2BzIaBNXv2bEJIamqq7MRx48bRpyuZBidOnGCelUgkZ86c0dfX9/f3V8lSOjo6MjIyrKysVq5caWFhQe+OZQeu0wwMDFpbW+m/R40aFR8fTwgJDg5ub2/PyMiQbRkTE+Pg4KDkb4exu3RCyMiRI//2t79duXLlhx9+kHuqsLBw7969c+bMGT16ND0FL8eALh2gv/p+Wh0Gt16NG7e2tj5+/PizZ88ePXr0wQcfWLf/vq0AACAASURBVFpaPnz4ULYBPVD52bNnzEDl+Ph4Zib0Fcrm5mZmSlRUFCHk2rVrSi7lzTffJIRs3bq1urq6qanp7NmzDg4OhJC0tDRmntOmTROLxSUlJZmZmQKB4Pbt2xRFVVVVDR8+3NnZOTU1ta6urqamZs+ePQYGBrIDrHosb0CX3uO4cdqGDRuEQmFUVFRhYaFEIiktLf3uu++sra09PT0bGxvlXi+8HH1eendwPRtUAn0I+kj5fdCTJ09Wr17t5OQkFAqtra3nzp1bVFTUXQOxWOzv73/mzBn6qaysLNmPmJ9++in13z8AMmPGDGWWUl1dHRkZaW9vLxQKLS0tFy1atGHDBnoOzKDfgoICLy8vQ0NDe3v7uLg45n/prxo7OzsLhUILC4upU6cyO3clyxugpdO8vLwUjxtnXL58ecGCBXQZxsbG7u7usbGxEolEweuFl6O3L0d3kNmgEjwKP4EEfZKUlBQWFob+A6AMvF9AJXA9GwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILMBAAC4AZkNAADADchsAAAAbkBmAwAAcAMyGwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILMBAAC4AZkNAADADchsAAAAbkBmAwAAcAMyGwAAgBsEbBcA3JaUlMR2CQAckJWVxXYJoA2Q2dAvYWFhbJcAADBY8CiKYrsGAFCZpKSksLAwvK8BtBKuZwMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHCDgO0CAKBfqqqq9u/fzzzMz88nhMTExDBTTE1Nly1bpv7CAEDleBRFsV0DAPRde3u7paVlfX29QPB/H8EpiuLxePTfEolk6dKl8fHx7BUIACqDc+MA3CYQCObOncvn8yX/0drayvxNCHnnnXfYrhEAVAPH2QCcl56e7uXl1eVTFhYWFRUVOjo6ai4JAAYCjrMBOG/SpEk2Njadp+vq6oaHhyOwAbQGMhuA83g83vz584VCodz01tbWefPmsVISAAwEnBsH0AZ5eXnjxo2Tmzhs2LDi4mI2ygGAAYHjbABtMHbs2BEjRshO0dXVXbRoEUvlAMCAQGYDaInw8HDZ0+Otra1hYWEs1gMAKodz4wBa4v79+yNGjKDf0Twez8XF5fr162wXBQCqhONsAC0xfPjwsWPH8vl8QohAIAgPD2e7IgBQMWQ2gPYIDw+nM7u9vR0nxgG0D86NA2iPiooKOzs7qVQ6ceLEjIwMtssBABXDcTaA9rC2tqZ/EG3hwoVs1wIAqofjbOC20NDQI0eOsF0FaLrExMQ5c+awXQVAf+FenMB57u7ua9asYbsKTfH8+fP4+HhsEFm4tA9aA5kNnGdnZ4dDKFl+fn52dnZsV6FBkNmgNXA9G0DbILABtBUyGwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILMBAAC4AZkNAADADchsAAAAbkBmAwAAcAMyGwAAgBuQ2QAAANyAzAYAAOAGZDYAAAA3ILNhMEpISODxeDweT09Pj+1a+iI1NXXkyJECgaL78inTRgEjIyOeDD6fb2pq6urqunz58tzc3L7NEwD6CZkNg9HcuXMpivL19WW7kF67f//+rFmzNm7cWFVV1Z82PWpsbLx27RohJDAwkKKotra2goKCv//97wUFBePHj3/vvfeampr6PHMA6BtkNgCXbNq0aeLEibm5ucbGxv1p01s6OjqWlpaBgYFnz55dv379/v37582bR1GUquYPAMro43kzAGDF999/r6+v3/82/REdHX3hwoVjx44lJCTMmzdv4BYEAHJwnA3AJcqE8YAGNiGEx+OtWLGCEPLtt98O6IIAQA4yGwaLgoKCoKAgsVhsaGjo5eWVnp7euU11dfXKlSsdHR11dXUtLCyCg4Pz8vLop5KTk5kBWcXFxWFhYSYmJubm5jNnzrx//z4zB4lE8vnnn48ePdrAwMDMzCwgIODYsWMdHR3KLIJDPD09CSHZ2dltbW30FGw6AHWgALgsJCQkJCSkx2Z37941MTGxtbU9depUQ0NDfn7+1KlTHR0dRSIR06a8vHzYsGGWlpYnTpxoaGi4efOmt7e3np5eZmYm0yYwMJAQEhgYmJmZ2djYmJaWpq+vP2HCBKbBkiVLxGLxqVOnmpqaKisr161bRwg5d+6c8otQkq2trY6OTp/b+Pj4mJmZZWVlKfh32TFocpqbm+kdSHl5OaXxm44QkpiY2GMzAM2HzAZuUzKzQ0NDCSFHjhxhppSVlYlEItnMXrhwISHk4MGDzJSKigqRSOTm5sZMoYMnJSVFtgBCSHV1Nf3Qyclp4sSJsoseOXIkEzzKLEJJ/cxsb29vU1NTxYGnILOZQeN0Zmv4pkNmg9bAuXEYFE6ePEkI8ff3Z6bY2NiMHDlStk1ycjKfz585cyYzxcrK6pVXXsnNzS0tLZVtOWHCBOZve3t7Qkh5eTn9cNq0aZmZmcuWLcvOzqbP6xYWFk6ePLm3ixho58+fr62t9fDw6Nu/V1RUEEKEQuELL7xABtmmA2ARMhu0n0QiaWho0NPTMzIykp0+dOhQ2Tb19fVSqVQsFsv+lsjVq1cJIXfv3pX9R7FYzPytq6tLCJFKpfTDuLi4AwcOPHjwwNfXd8iQIdOmTfvtt9/6sAgNR48G8PDwEAqF2HQAaoPMBu0nEomMjY1bWloaGxtlp9fW1sq2MTExEQgEbW1tnc9H+fj4KLksHo+3YMGC06dP19XVJScnUxQVHBy8Y8cOFS6CdVKpNC4ujhDy4YcfEmw6ADVCZsOgMH36dPKfM+S0J0+eFBYWyrYJDg5ub2/PyMiQnRgTE+Pg4NDe3q7kgkxMTAoKCgghQqHQz8+PHjJ94sQJFS6CdRs3brx8+fLs2bPpUQIEmw5AXZDZMCh8+eWXZmZmq1evTktLa2xsvH379vz58+VOlf/jH/8YPnz44sWLf//99/r6+tra2r179/7973/fvn17r361+/3338/Pz5dIJI8fP966dStFUW+++aZqF9F/b775prm5eXZ2tpLtpVLp48ePjx496uvru3Xr1sWLFx88eJDH49HPDqpNB8Cm/g1hA2CZkuPGKYoqLCwMCgoaMmQI/RWj48ePM783HhERQbepqalZu3ats7OzUCi0sLCYOnVqWloa/VRWVpbsG+fTTz+l/vuXO2fMmEFRVF5eXmRk5EsvvUR/ydjd3X3fvn1SqZQpQ8EilJGSktL5Xbxv377etvHy8lI8btzQ0FD233k8nlgsdnFx+eCDD3Jzczu31+RNRzBuHLQFj8IvBgOX0adnDx8+zHYhoLl4PF5iYuKcOXPYLgSgv3BuHAAAgBuQ2QAAANyAzAbQFLzubd68me3qAIB9GG8JoCkwuAQAFMNxNgAAADcgswEAALgBmQ0AAMANyGwAAABuQGYDAABwAzIbAACAG5DZAAAA3IDMBgAA4AZkNgAAADcgswEAALgBmQ0AAMANyGwAAABuQGYDAABwA+7rBZx35MgRHo/HdhUAAAOOh9v/AadlZWU9evSI7Sr6rqCgICYm5sUXX/zkk0808JMHRVH/+Mc/7t69+/HHH7/88stsl9N3EydOtLOzY7sKgP5CZgOw5syZM0FBQT4+PklJSXp6emyX07XW1tYFCxYcPXr04MGDb7/9NtvlAAxquJ4NwI7k5OQZM2YEBgb++uuvGhvYhBBdXd1ffvll0aJFYWFh33//PdvlAAxquJ4NwIIDBw5ERERERkbu2rWLz9f0j846Ojp79uxxcnJaunTp06dP161bx3ZFAIMUMhtA3Xbt2rV69er169dHR0ezXUsvREVFGRgYrF69+smTJ9yqHEBrILMB1ComJmbjxo3btm3761//ynYtvfbRRx+Zmpq+99579fX1cXFxmn+GAEDLILMB1ISiqLVr13799df79u2LiIhgu5w+mj9/vlgsDgsLq6ysPHTokCZfiQfQPhg3DqAOHR0dS5Ys+eWXXw4ePBgSEsJ2Of114cKFWbNmjR8/Pjk52djYmO1yAAYLZDbAgJNIJO+8887Jkyd//fVXf39/tstRjdzc3LfeemvYsGGpqakvvPAC2+UADArIbICB9fz589mzZ+fk5Bw/fnzSpElsl6NKDx488PPz09PT++OPP/CLJQBqgCEkAAPo6dOnfn5++fn5586d07LAJoQ4Ozunp6fr6Oh4eXndvXuX7XIAtB8yG2CgVFZWTp48uays7OLFi2PHjmW7nAFhbW194cIFa2trLy+vvLw8tssB0HLIbIABUVxc7OXl1dramp6ePnLkSLbLGUCmpqZpaWmurq4+Pj6XLl1iuxwAbYbMBlC9O3fueHp6Dhky5OLFi/b29myXM+AMDQ1TUlL8/Pz8/Px+++03tssB0FrIbAAVu3LlyhtvvOHs7Hz27FkLCwu2y1ETXV3dQ4cOhYeHh4aG/utf/2K7HADthN9UAVAl+ovLkyZNOnLkiIGBAdvlqJWOjs7evXttbGwiIiKePn26du1atisC0DY6mzdvZrsGAC1x/PjxoKAgPz+/I0eO6Ovrs10OC3g83uTJk01MTP761782NzdPmTKF7YoAtAqOswFUg75h5bx5877//nuBYFC/s1atWmVqahoREdHQ0PD111/jZ8kBVGVQ71kAVGX37t0rVqxYsWLFP//5Tx6Px3Y57AsPDzcxMQkLC3v69OmPP/4oFArZrghAG+DzL0B/xcTELF++/OOPP46NjUVgM2bNmvX777+fOHFi+vTpjY2NbJcDoA3w26UAfUdRVFRU1Pbt23fs2LF69Wq2y9FEubm506dPd3JySk1NNTc3Z7scAG5DZgP0UUdHxwcffPDDDz989913ixYtYrsczVVQUDB16lRjY+NTp07Z2tqyXQ4AhyGzAfqitbV1wYIFR48e/eWXX4KDg9kuR9OVlJRMnTpVIpGcOnVqxIgRbJcDwFW4ng3Qa01NTUFBQSdOnEhJSUFgK8PBwSEzM9PKyuqNN97Az5ID9BkyG6B36uvr/f39s7OzT58+7efnx3Y5nGFmZpaWlubi4uLj45Oens52OQCchMwG6IXHjx/7+Pjcu3fv/Pnz7u7ubJfDMUZGRikpKVOmTPH39//999/ZLgeAe5DZAMoqLy/39fV9+vTppUuXxowZw3Y5nCQSiRISEt55551Zs2bt37+f7XIAOAa/qQKglAcPHvj5+YlEovT0dAx+7g8dHZ34+HhTU9PFixc/ffp0zZo1bFcEwBn4vXGAnt26devNN98cOnTomTNnrKys2C6H83g8np+fn56e3vr16/Gz5ADKw3E2QA8uX7781ltvvfrqq8eOHRsyZAjb5WiPqKgoKyurJUuWNDY27tq1Cz9LDtAjZDaAImfPng0KCvL29k5KShqct+oaUAsXLhSLxfPmzXv69On+/fvxs+QAiuGDLUC3jh49OmPGjICAgF9//RWBPUCCgoJSU1NTUlJmz57d3NzMdjkAGg2ZDdC1n376KSQk5L333vvpp59w/DegfHx8zp49++9//9vHx6e2tpbtcgA0FzIboAvffPPNokWL/vrXv3777be4zqoG48ePv3jxYnl5ube3d3l5OdvlAGgo7IwA5MXExKxcuTI6Ojo6OprtWgaRl1566dKlS62trZ6envfu3WO7HABNhMyGwSsuLk4ikchOoShq7dq1n3766d69ez/++GO2Chu0hg0bdunSJRMTEy8vr/z8fLlnr169mpOTw0phABoC9/WCQSo/P3/s2LEzZ8789ddfBQIBIaSjo2PZsmU///zzTz/9NGfOHLYLHLwaGxtnz56dm5t7/PjxiRMn0hOLioo8PDxeeeWVixcvslseAItwnA2D1CeffKKjo5Oamvree+9RFNXa2hoWFpaQkHD06FEENruMjIyOHz/u4+MzderUkydPEkJKS0t9fHzq6+svXbr0xx9/sF0gAGtwnA2DUU5Ozuuvv053fj6fv3jx4pKSkn//+9/Hjx/39PRkuzogROa0R1xc3NatW4uLi9va2nR0dEaNGnXjxg0MDITBCZkNg5G3t3dmZmZ7ezv9kMfjGRsbnz9/fty4cewWBrIoilq5cuWBAweam5vb2troiTwe79ChQ2FhYezWBsAKfFaFQef06dMXL15kApsQQlHUs2fPzpw5w2JV0FlbW1tBQUFTUxMT2IQQHo+3YcMG2SkAgwcyGwadjRs30oPO5Kxfv/67775Tfz3QJalU+u67754/f1720xU9vaSk5IcffmCrMAAW4dw4DC5Hjx4NCgrq7lk+n5+UlPT222+rsyTojKKoyMjIH374oaOjo/OzPB7PwsKiuLgYPygLgw2Os2EQkUqlGzZs0NHR6fJZoVBIUdShQ4dw3pV1V65cSU5Opiiqy7FmFEXV1NR8/fXX6i8MgF3IbBhEfvnll8LCws6HbgKBQCgUhoWF3bp168iRI/h1cdZNmDChtLQ0ISGBHhXY+RXp6OjYsmULfpwcBhucG4fBoq2t7cUXXywtLZVKpfQUPp9PUZS5ufmHH3740UcfmZubs1shdCk3N3fnzp0JCQl8Pl/2FIhAIFi/fv3//M//sFgbgJohs2Gw2Lt37/Lly+nAFgqFbW1tLi4u69atmzdvHg6sNV95eXl8fPw///nPxsZGqVRK77h0dXUfPHhga2vLdnUAaoLMhkGhpaXF0dGxqqpKIBBIpdLAwMB169Yxv4sJXPH8+fOffvppx44dd+/eFQgE7e3t77///u7du9muC0BNkNkcEBoaynYJnFdUVJSfny8QCJydnYcPH25oaMh2RYp4eHisXbu2nzPZsWNHVlaWSurRQFVVVUVFRVVVVTwez9/f38jIiO2KBrXDhw+zXcJg0cW3VEHTHDlyxN3d3c7Oju1CuKq9vb20tNTV1dXJyanLb2ZrlOzsbJXMJysrKzs7293dXSVz0zSWlpaWlpYNDQ337t0rKip67bXX2K5okCotLVVVjwVlaPr+C2hr1qzBjSv6rLGx0cDAgCu/UK3C0yru7u6D4QDo2bNnxsbGPB6P7UIGo6SkJPyOrDohs0H74cSpdhsyZAjbJQCoCTeOPAAAAACZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhs4z8jIiCeDz+ebmpq6urouX748NzeX7eo4JiEhgd6Menp6bNeitdBjoc+Q2UAIIY2NjSNGjJg5c6YGzk2ZxV27do0QEhgYSFFUW1tbQUHB3//+94KCgvHjx7/33ntNTU3qqUQLzJ07l6IoX19ftgvRZuix0GfIbCCEEIqipFKpVCoduLkZGRl5enqqZP6K6ejoWFpaBgYGnj17dv369fv37583bx5FUWpYNAwctfUf5amqJPRYUB4yGwghxNjY+P79+6mpqRo4t/6Ijo5+/fXXjx07lpCQwHYtAD1DjwXFkNmgzXg83ooVKwgh3377Ldu1APQMPRYUQ2Zrg+3bt9ODWezs7HJycnx9fY2NjQ0MDHx8fDIyMug2ycnJzJiXwsLCOXPmmJub0w+/++475qmWlhZmtjU1NWvXrh0+fLiurq6pqen06dPPnTvXh7nR5T1//jwjI4OeLhAI6urqZIfhfPHFF4SQ9vZ2ZkpISIhKNg599jI7O7utrY2eUl1dvXLlSkdHR11dXQsLi+Dg4Ly8vM7rVVxcHBYWZmJiYm5uPnPmzPv37zPzlEgkn3/++ejRow0MDMzMzAICAo4dO9bR0cE0ULAIDVRQUBAUFCQWiw0NDb28vNLT0zu3UbBGynQ/Qkh7e3tiYqKfn5+VlZW+vr6Li0tsbCxzAUVBj9qwYUPn/iP3Lw8fPgwLCzM2NjY3N1+wYMHTp0+Li4sDAgKMjY2tra2XLl3a0NCg5Ooo0we67NKqejnQY0ERCjQeISQxMbHHZq6uroaGhh4eHpmZmY2NjTk5OWPGjNHV1T1//jzTJjAwkBDi7e197ty558+fZ2dn6+joVFdXM081NzfTLSsqKpycnCwtLVNSUurr6wsLC4ODg3k83r59+/owN4qiDA0NJ02aJFezv78/n8+/d++e7EQPD4+DBw8yD318fMzMzLKyshSsu+yIHjnNzc10Vy8vL6coqry8fNiwYZaWlidOnGhoaLh586a3t7eenl5mZqbcegUGBtJbMi0tTV9ff8KECUyDJUuWiMXiU6dONTU1VVZWrlu3jhBy7tw5+lllFqFASEhISEiIMi1VMp+7d++amJjY2tqeOnWqoaEhPz9/6tSpjo6OIpGIaaPMGvXY/VJSUgghX375ZW1tbXV19a5du/h8/rp162SLUdCjuuw/zL8EBwdfuXKlsbHxwIEDhJDp06cHBgZeu3atoaFhz549hJA1a9b0anV67AMKShpUPTYxMRE5ok7Y1hygfGYTQq5du8ZMyc/PJ4S4uroyU+j3dmpqaud/l0vZRYsWEUIOHTrENGhpabGxsdHX16+srOzt3KhudnB//PEHIWT58uXMlPT0dFtb29bWVmaKt7e3qamp4t2Hgj0gMwSX3gMuXLiQECL7maCiokIkErm5uckVn5KSwkyhD/rp8KAoysnJaeLEibJLGTlyJLMHVGYRCqg5s0NDQwkhR44cYaaUlZWJRCLZzFZmjXrsfikpKZMnT5Zd9Pz584VCYX19PTNFQY9SnNknTpxgprzyyiuEkAsXLjBTnJycRo0a1avV6bEPKChpUPVYZLaaYVtzQK+Os+Um2tjYMG9+6j/v7SdPnnT+d7mUFYvFhJBnz57JtlmwYAEh5Mcff+zt3Kjud3AuLi4GBgbMTAIDA6Ojo3tcWTkK9oD0GUKhUEh/DhCLxXw+XzYnKIp67bXXCCGPHj2SLZ75aEJR1Jo1awgh169fpx9+8MEHhJClS5dmZWW1t7fLLVGZRSig5sw2NjYmhDQ0NMhOdHFxkc1sZdZIme4nZ9u2bYSQzseLXfYoxZldVVXFTPHz8yOEPH/+nJni6elpbGzcq9XpsQ8oKEkZWtNjkdlqhuvZWsXExERuytChQwkhjx8/lp1oaGioeD4SiaS+vl5PT4/eoTMsLS0JIZWVlb2am2KrV69uamqiR9wUFRWdPXt22bJl/ZmhHPrqrIeHh1AopNdLKpWKxWLZq+lXr14lhNy9e1f2H+lPLTRdXV1CCHPxNS4u7sCBAw8ePPD19R0yZMi0adN+++03+qleLYJ1EomkoaFBT0/PyMhIdjrdbZg2Sq6R4u5XX1//+eefu7i4mJqa0nP4+OOPCSGdv4vchx41ZMgQ5m8+n6+jo2NgYMBM0dHRYV47VfWBgYMeCwogs7VKTU0N9d9f66R3l7K7YGWIRCKxWNzS0iI3cqeqqooQYmVl1YfaeDxel9PfffddS0vLb775RiKRfPXVVwsXLjQ1Ne3D/LsklUrj4uIIIR9++CEhRCQSmZiYCASCtra2zh9gfXx8lF+XBQsWnD59uq6uLjk5maKo4ODgHTt2qHAR6iESiYyNjVtaWhobG2Wn19bWyrZRco0Ud7+AgIAtW7YsXbq0qKhIKpVSFLVz505CCKXcF5G76z+9pcIXSFUlyUKPBcWQ2VqlpaUlJyeHeXjjxo3y8nJXV1dra+vezmr27NmEkBMnTjBTJBLJmTNn9PX1/f39+1CbgYFBa2sr/feoUaPi4+Ppv0Ui0fLlyx8/fvzVV18dPHhw1apVfZh5dzZu3Hj58uXZs2fTV20JIcHBwe3t7bLjmQkhMTExDg4O7e3tSs7WxMSkoKCAECIUCv38/Oixu8y2Uski1Gb69OmEkJMnTzJTnjx5UlhYKNtGyTVS0P06OjoyMjKsrKxWrlxpYWFBpx0z2EoZ3fWfPlDVC6TCkhjosdADRSfOQTMQpa9ni8ViX1/fHseNy15m7u4p2XHjz549Y8aNx8fH92FuFEVNmzZNLBaXlJRkZmYKBILbt28zT1VXV+vr6/N4vC4v7/V2FG5HR0dVVVVycvKbb75JCFm8eHFTUxPTsqqqavjw4c7OzqmpqXV1dTU1NXv27DEwMJDdwp2Lj4qKIjIDrMRisbe39/Xr11taWqqqqjZv3kwI+eKLL5RfhAJqvp597949MzMzZtz4rVu3/P39hw4dKns9W5k16rH70S/H1q1bq6urm5qazp496+DgQAhJS0tjZqKgR3XXfzr/i7+/v46Ojuz/ent7y15rV0kfUFDSoOqxuJ6tZtjWHKB8Ztva2t6+fdvf39/Y2FhfX9/b2zs9PZ1+Nisrq7uPa8yVLdq7775LT3/y5Mnq1audnJyEQqFYLPb39z9z5kyf51ZQUODl5WVoaGhvbx8XFydX/NKlS8l/j/VleHl5KR6FK3f5k8fjicViFxeXDz74IDc3t3N7+nvnzs7OQqHQwsJi6tSpTGzIrdenn35K/feZ2xkzZlAUlZeXFxkZ+dJLL9HfdnV3d9+3bx99vrfHRfRIzZlNUVRhYWFQUNCQIUPoLwgdP36c+b3xiIgIuk2Pa6S4+1EUVV1dHRkZaW9vLxQKLS0tFy1atGHDBnopbm5uCnoUrXP/6fxiyR7lE0L+8Y9/XLp0SXbK3/72tx5XR8k+0GVJtEHVY5HZasaj8Ku2Go/H4yUmJs6ZM0dxs7Fjxz558qS0tFQ9VanWv/71r7i4uCtXrrBdCPvok6KHDx/WkPkoidPdD/osKSkpLCwMOaI2uJ4NGmHPnj1r165luwoAXiXmTAAAAS9JREFUAI2GzAbWfPfdd7Nnz25sbNyzZ8/Tp097PJEAADDIIbO1Af3rx9evXy8rK+PxeJ999hnbFSkrOTnZ1NR09+7dCQkJKvzFZlAn7nY/AM7BXlIbrFu3jv4BYW5ZsmTJkiVL2K4C+ouj3Q+Ai3CcDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgMwGAADgBmQ2AAAANyCzAQAAuAGZDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAbkNkAAADcgPt6ccPOnTsPHz7MdhWgDtnZ2e7u7qqaVWhoqEpmBdCl0tJStksYXJDZHBASEsJ2CaA+7u7uHh4e/Z+PSmYCoJidnR12UOrEoyiK7RoAAACgZ7ieDQAAwA3IbAAAAG5AZgMAAHADMhsAAIAb/h8iPxs/N92NoQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 905 ms (started: 2021-07-27 16:30:18 +08:00)\n"
     ]
    }
   ],
   "source": [
    "keras.utils.plot_model(model, \"ticket_classifier.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "intended-causing",
   "metadata": {},
   "source": [
    "您可以将模型中每一层的输入和输出形状添加到此图中，这在调试过程中会有所帮助："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "legitimate-console",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABZEAAAGVCAIAAAAACyiUAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzde1wTV94/8JMQLiGEoIAQASvYQleq0QIiCouAC1qoFx6QWu1tF1+s3RaptSuobZ9Xq7W11MputVVp9+mrYoXaR1+rVVsXb+WiBhSsVdRFVBCCXOQqt8D8/ji/zjObQAgQMgE+77/IycnMmZnM9wzfnDkjYBiGAAAAAAAAAACYGCHfDQAAAAAAAAAA6AVyFgAAAAAAAABgipCzAAAAAAAAAABThJwFAAAAAAAAAJgiEfdFfn7+9u3b+WoKAMBI9N133w19Idu3b8/Pzx/6cgAAwHSsXbs2ICBgiAvB9TkAjDUBAQFr165lX/7HOIvy8vKDBw8avUkwAlRUVOC7wXX+/Pnz58/z3QrgmQHPi/z8fHyjYOQ6ePBgRUUF360wFegxgTp48GB5efnQl4Prc+gXrks1oFca0c6fP6/xS55Iu5JBfjOEUSYrKysuLg7fDVZsbCzByTLm0fPCUEubPXs2vlEwQgkEgjfeeGPZsmV8N8QkoMcESiAQGHBp+EaBDrgu1YBeaUSj32cuzGcBAAAAAAAAAKYIOQsAAAAAAAAAMEXIWQAAAAAAAACAKULOAgAAAAAAAABMEXIWAMNo3759gt/Y2NhovHv37t1FixY1NTXV1tay1WbOnNne3s6txn1XIBD4+voacQsG5tixY56eniJRL5P7UkVFRZGRkXZ2dlKpdP78+bm5uXzVSU5OzszM1PhIcnIyu59nz56t1zYDAAwKOggN6CAAwJgQhDWYcvBEzgKGV0tLyxNPPBEVFcV3Q/j0+eefMwzT0tLCLSwqKvL19Q0PD7e1tXVwcGAYRqlU0vKkpCRuTfpufn6+vb09wzAFBQVGbb1+SktLFy1alJKSUl1d3VedCxcuzJkzRyqVXr9+vayszMPDY968eT/99BMvdVatWpWSkvL2229zP/Xhhx8yDMMwjJmZ2ZB2BwDoAR0EQQfxG3QQALxAHEYQpkw9eDIcNC/CAGgZ9HejqanJw8Nj4cKFBm+SniQSydy5cw2+2JiYmJiYmH6rffPNN+S3aMjV2Njo6uqakJDALVQqlZaWlvb29oSQ/fv3a3yEjYamafny5Vu3bu3q6nJxcTEzM9Ou0N3d7e3tLZfLHz16REvUarWXl5ebm1t7e7vx6zAMU1RUJBAIMjMztVtrZmbm7+/f71YbMGbq+Y0CME2EkF5PJd1GawehZ2RAB8EalR0EM9jzQhuuz6Ffg76KGK1xWJ+zD0GYZWrBU/v7jHEWMLykUmlpaemxY8f4bohp2bZtm0qleueddzTKraysMjIyhEJhQkLCzZs3eWnb4Hz55ZfJyck6xpudO3fu119/jYmJEYvFtMTMzGz58uXl5eVHjx41fh1CiEKhiImJefPNN9VqtcF2BADoDR1Er9BBEHQQAMaCOKwNQZiYXvBEzgLA2BiGSU9P9/f3nzhxova7ERERmzZtam5ujo2N1bhlzpSx8asvp06dIoRo3OZHX2ZnZxu/DrV06dKKiooffvihv+0DADAGdBAsdBAAYHwIwiyTCp7IWcAwOnz4MDvnCj2xuSV37tyJi4uzs7Ozt7ePiooqLS2ln0pNTaUVXF1dlUplWFiYVCq1trYOCQlhJ3rZvHkzrRMYGEhLTpw4QUscHBy4y2ltbc3NzaVv6cgvGlNxcXF1dbVCoeirwrvvvhseHn7lypXXX39d96Lq6urWrl07ZcoUCwuLcePGLVy48PTp0/QtfXY1VVNTk5iYOHnyZAsLC0dHx+jo6KKioqFvpoaSkhJCiKurK7fQxcWFEMLmqo1Zh5oxYwYh5McffxzClgHAYKCD6BU6CBY6CIDhhjisDUGYZVLBEzkLGEZLlixhGGbx4sW9liQlJSUlJd2/fz8zM/PUqVPLly+nddatW8cwjEKhaGhoWLNmzebNm1Uq1blz5+rr60NDQ8+ePUsI2bRpE8MwEomEXfKCBQsYhvHx8WFL6HK4t8lxBymFhoba29ufP39+uHeCtqtXrxKtc55LKBRmZGS4ubmlp6dnZGT0VU2lUvn5+e3fvz8tLa22tvbChQvW1tZhYWHp6elEv11NCKmqqvLz88vKytq1a1d9ff2ZM2fq6+sDAgLy8/MNuc2ENDQ0EEK4h4wQQmdpfvjwofHrUDTU0iMCAMaEDqJX6CBY6CAAhhvisDYEYZZJBU/kLIA38fHxAQEBEolk/vz5kZGRSqWytraWW6G1tXXXrl20jq+v7759+zo7O9esWWOQtff09ND4aJClDUhVVRUhRCaT6ajj4OCQlZVlbm6ekJBA85raUlJSysrKduzYERUVZWtr6+npuX//frlcnpiYqDEtsI5dnZKScvfu3e3btz/zzDM2Njbe3t4HDhxgGKbf5LFB0P0vEAj4qmNraysQCOgRAQDTgQ5CRx10EMapgw4CxrixGYcRhFkmFTyRswDe+Pn5sX+7ubkRQiorK7kVJBIJHVxETZs2beLEicXFxQY5B9hU5dAXNVB09J25ubnuarNnz05NTW1tbY2NjW1ra9OucOjQIUJIZGQkW2JpaRkWFtbW1qYxHEvHrj58+LBQKOQ+48rZ2dnb27uwsLCiomKgm6aDnZ0dIaS1tZVbSF/St4xchyUSiXrdvQDAI3QQuquhgxjWOix0EDCWjc04jCDMMqngiZwF8IabwrSwsCCE9PT0cCtonwATJkwghDx48GD4WzeMrKysCCFdXV391kxMTIyLi7t69eprr72m8VZHR0djY6OVlZVUKuWWOzk5EUJUKhW3sK9dTRfS09Mjk8kEHJcuXSKE3Lp1a3Ab2Ksnn3ySEKIRYe/fv08I8fT0NH4dllqt7ndqIgAwMnQQ/dZEBzF8dVjoIGAsG5txGEGYZVLBEzkLMF11dXUaQ8JoEKQBkRAiFAo7Ozu5FeitVly6RzTxQi6XE0IaGxv1qZyenu7l5fXVV1/Rh0izLC0tZTJZe3t7c3Mzt5yON3N2dtZn4ZaWlnZ2diKRqKurS/vZyCEhIfpukh7o0goLC7mF9GVYWJjx61BNTU0Mw9AjAgAjCDoIgg4CHQQAr0ZlHEYQZplU8ETOAkxXe3u7UqlkX/7yyy+VlZUKhYI9B+RyOU3vUSqV6t69exoLsba2ZsOll5fXnj17hrnV/XvqqaeIVp6yLzY2Nt9//71EItm1a5fGW0uXLiWEcJ8k1NHRkZ2dLRaLIyIi9GxMdHS0Wq1m53mmPvroo0mTJhn2ucrBwcFTp049ePAg+2io7u7uAwcOuLm5sQPnjFmHot8fekQAYARBB0HQQaCDAODVqIzDCMK0xNSCJ3IWYLpkMtmGDRvy8/NbW1sLCgpWrlxpYWGRlpbGVggPD6+srPzss89aWlpKS0vXrFnDZnZZTz/99M2bN8vLy/Pz82/fvh0UFETLeZwWXqFQTJgwobi4WM/63t7eu3fv1i7funWru7t7UlLS0aNHm5ubb968+fzzz1dVVaWlpdGxZ/rYunXrlClT/vjHPx4/fryxsbG+vn737t3vvfdeamoq+8SplStXCgSCsrIyPZfZK6FQ+OWXX9bX17/yyisqlaquru4vf/nLrVu39u7dS4fhGbkORZ8XFR4ePpRNAwDjQwdBoYNABwHAl1EZhxGETTR4cgeZZGZmapQAUIP7btDpZ1grVqzQeDbPxo0bmf8cVBYZGUk/q1AoXFxcrl27FhERIZVKxWJxcHBwTk4Od/kNDQ3x8fFyuVwsFgcGBiqVSvYRSuvXr6d1SkpKgoKCJBKJm5vbzp072c8GBQWNGzcuLy9vcDskJiYmJiam32p0qNjnn3+uUb5hwwaRSHT//n36sqamhrsTfHx8tBe1evVqe3t7jcLa2tqkpCR3d3dzc3OZTBYREZGdnU3f0n9X08dHe3h4mJubOzo6hoeHnzx5kruW0NBQGxsbtVqtY0uPHDmiHV727t2rUe3SpUsLFy60tbW1sbEJDQ3VOKDGrxMbG+vi4tLZ2alRbmZm5u/vr2N7KQPGTD2/UQCmiRCSmZk5oI+M4g5Cz8iADkKj2ijrIJhBnRe9wvU59GtwVxGjOA7rc/YhCGtUM53gqf19Rs4C9GL87wYNhcZc44AMMWfR0NDg4uKSkJAwPK0zpIcPH4rF4vj4eL4bYnhFRUUCgeDbb7/Vfgs5C4ABMdT/Znoy8Q5iiDkLdBCmYOgdBIOcBRiR8a8iTDwODyVngSA8FAYJntrfZ9wbAsADmUx25MiRgwcP7ty5k++26MIwTGJioq2t7fvvv893Wwzs9u3b0dHRKSkpzz33HN9tAQD4P+ggeIcOAmAsQxAetOELnjzkLFJTU+mTWlxdXQ27ZBsbG+6TYFJTUw27/KEw5bbBcFu9erVAILCxseEWzpw5s6Cg4Pjx401NTXw1rF/V1dW3b9/Ozs7Wc4rjEWT37t1btmzZsmULtzA5OZmeod3d3Xw1bAQZpmB+4MABuliN2yOHzpTjsCm3DYYVOggThA5iOJhylDPltsFwQxA2oGEMntxBF0Mce9bc3Pz444+zt9/oNkwDii5fvkwIWbx4scGXPHSm3LZ+GXNc4scff8z9itJbvEwNRvIDg3tDGIYZtmAeFhZmaWlp8MWachw25bb1ixjr3pAR0UFgJD9QhjovRs03ypSjnCm3TR/GvIoYEXHYaL0SDAeD3RtiY2MTGBionf7o6enp6enpt+boNgY32bDWrVvH/Y5u3ryZ7xYBmJDhizCIXcaB/TwU6CAAdEB46Rd20dAhDoPxiQy4LKlUWlpaasAFAgAAAAAAAMCYhTk4AQAAAAAAAMAUDThnQSdda21tzc3NpdNpiEQiQsjhw4fZqWva29t11OxLTU1NYmLi5MmTLSwsHB0do6Oji4qKBr1hLG7D7ty5ExcXZ2dnZ29vHxUVxY4K4c4kp1Qqw8LCpFKptbV1SEhIbm4urbN582Zahx1RduLECVri4OCge+foT61WZ2Zm/uEPf3B2dhaLxdOmTUtLS6O32zQ0NHDnB6IDsdRqNVsSExNDF6JjT3L3xo0bN5YtW2Zvb09f1tbWDmlHA8Aw0x1hdJz4gYGB7Im/cuVKQsj8+fPZkoaGhqHHrpKSksjISJlMphE5Kfp08SlTplhYWIwbN27hwoWnT5/W+PiSJUtkMplEIgkKCsrJyWHf0j/0DQ76CPQRAKOAjvCiI26w2CBsbW09a9aso0ePst1EfHw8IaSjo+Odd9558sknra2tx48f/+yzz/7zn/8c+nSkiMCIwAB64d6PpP8cPxKJZO7cudrlixcvJoS0tbX1W1Nj2rbKysrHHnvMycnphx9+aG5uvnr1anBwsJWVVV5eHlsnJCRk/Pjx+fn5OhrW1ww6tGGLFy/Oy8traWk5efKkWCz28/PTaJJEIgkICKB1lErl9OnTLSwszpw5o2NzfHx87O3t9dk5+szuc+TIEULIBx98UF9fX1NT87e//U0oFHJvG4uIiBAKhf/+97+5nwoICMjIyKB/67Mn6d4IDg4+ffp0a2vr+fPnzczMampqdDRs1Mz/ZCgjd8ZEMCBe5uDsNcL0e+IXFRVJJBKFQtHS0sIwTHt7u7+/v8bTs/uKXbopFAqZTBYSEpKTk9Pc3KwdOauqqtzd3Z2cnI4cOdLY2Hjjxo3o6GiBQLB3715a4datW3Z2di4uLj/99FNzc/OVK1fCw8MnT57MnYOz39DHoI9gGIa/PoJgtjMO9JhAGeq8GOL1eb9xQyMIX716df78+Y6OjtwgHB8fL5PJfvrpp0ePHqlUqnXr1hFCTp8+zVZABGb4i8AMrku1oFca0bS/z6aSs3jppZcIIdyrz6qqKktLSx8fH7YkODh43Lhx3LNam+5oeOTIEbaE5ju5IUChUBBCLl++zJZcuXKFEKJQKHRsjsGj4bx587glK1euNDc3b2xspC9//PFHQsirr77KVsjJyXFxcens7KQv9dmTdG8cO3ZMR0s04ApMA/oGYEwpZ6HPiZ+VlUUIiY6O7unpeemllzZs2KDPkvtFIyf3OlUjcr788suEEG5+pL29feLEiWKxWKVSMQwTGxtLCDl48CBb4f79+5aWltzL5X5DH4M+gmEY/voIXB1yoccEynRyFrrjhnYQfvDggbW1NTcIu7u7z5kzh7sQT09Pbs4CEZjhLwIzuC7Vgl5pRDPdnIVMJhMKhew5Tz399NOEkPLycn2aROmOhvT6mHrjjTcIIcXFxdwmSSQSjQ9OnDiREFJZWdnX5hg2GmqjzxPi9gHTpk2ztraura1lN+3DDz9k39VnT9K9wS5BH/S7AQDaBnRG92WIOQs9Q+jGjRsJIXPmzImKiuru7tZnyf1SKBRWVlY9PT3cQm7klMlkhJCmpiZuhRdeeIEQ8vXXXzMMI5VKCSHNzc3cCtOmTdN41qnu0Kcn9BHD1EfwceYBjACmkLPQphE3eg3CTz/9NDcIr169mhCyatWq/Px8tVo9kOb/H0TgYYrAzG9ZHoBRQ+Oq2JDPDRm0jo6OxsZGQgi9tNVw69YtV1dXg6yIu3wLCwtCiMbtfHZ2dhofmTBhQmVl5YMHD+RyuUHaoFtjY+Mnn3xy6NChioqKhoYGtvzRo0fs30lJSX/605927dr19ttv37x589SpU//4xz/oWwPakxKJZKDNQ+aC9emnnxJCaJ8KY1Z+fv6OHTv4bsUATvz333//X//6V15e3tdffy0UGmwOZnrDLbeEjZzjx49vbGy0srKi18QsJycnQohKpero6GhubraysrKxsdFYws2bN7klOkKfoaCPGEofkZSUFBAQMKCPjFY0MqDHhLi4OL6bQEh/caOvIDxu3Djuy507dwYEBHz99ddhYWGEkKCgoISEhKVLlxqwnYjAQ7xKnz17Nq5LWXFxceiVRi76fxbXIHMWGpenQ6xpaWlpZ2fX0tLS1tY20OlwDKuuro5hGG6bHzx4QAiZMGECfSkUCjs7O7kf4cYsSv+do+3ZZ5/9+eef09LSli9f7uDgIBAIduzY8cYbbzCcX7FWrFixYcOGzz777K9//esnn3zy0ksvsf3KcO/JZcuWGXyZI9R3331HsEOAEOPnLLQjjP4n/pkzZxobG6dNm/bqq68qFAo61FbHkvVEL8K42MhpaWkpk8kaGxubm5u5aYvq6mpCiLOzs6WlpVQqbW5ubmlp4V4x19fXayxTR+gzGvQROgQEBCAksnbs2IG9AcbPWfQaXnTHjb6CMA1u3CW/8MILL7zwQldX15kzZ1JTU6Ojoz/55JO1a9cO90axEIF1c3V1RdhhxcXFoVcauej/WVyD/J3N2tqaDQpeXl579uwZYs3o6Gi1Wq0x1fxHH300adIktVo9uEYOQnt7u1KpZF/+8ssvlZWVCoWCTd/K5fL79++zFVQq1b179zQWov/OYYlEopKSku7u7tzcXGdn58TEREdHRxpV29raNCpbWlq++uqrDx48+OSTTzIyMtasWcN910T2JAAMk14jjD4nfllZ2Z/+9Kfvv//+n//8p1gsXrx4cU1NTb9L1kdLS0txcTH7UiNy0h/ifvjhB7ZCR0dHdna2WCyOiIgghCxcuJAQcuLECbZCbW3tjRs3NNaiO/QZB/oIADBl2uFFn7ihHYRVKpXGSDc7O7uSkhJCiLm5+R/+8Af6hAtuYDcCRGCAMWuQOYunn3765s2b5eXl+fn5t2/fDgoKGmLNrVu3Tpky5Y9//OPx48cbGxvr6+t379793nvvpaamspnI0NBQe3v78+fPD67N+pDJZBs2bMjPz29tbS0oKFi5cqWFhUVaWhpbITw8vLKy8rPPPmtpaSktLV2zZg2b3GXpv3M0mJmZzZs3T6VSffzxx7W1tW1tbadPn/7iiy+0a7766qtisXjTpk3z589//PHHuW/psycBYOTqNcL0e+K3tLQsWbJkx44dU6dOnTx58sGDBysrK2NiYrq6unQvWR8SieS11167cOFCr5Fz69at7u7uSUlJR48ebW5uvnnz5vPPP19VVZWWlkbvEPnggw/Gjx+flJR08uTJlpaWa9eurVy5UmOUMqUj9BH0ERzoIwDGJu3wok/c0AjCV69efeWVV5ydnTUW/uc///nKlSsdHR0PHjzYtm0bwzChoaHsu4jALERgAMPjTm6h/xw/JSUlQUFBEonEzc1t586dDMMcOnSIu9gVK1b0VZNOV8PauHEjrVlXV7d27VoPDw9zc3NHR8fw8PCTJ09yVxoUFKR7RmKNW78+/vhjhmHy8/O1V8ctiYyMpB+n04Jeu3YtIiJCKpWKxeLg4OCcnBzuKhoaGuLj4+VyuVgsDgwMVCqVPj4+dDnr16/va5O126bt+vXrDMPU1NQkJCS4ubmZm5s7OTm9/PLLycnJtAJ3SmGGYVatWkUIOXv2rPZ+0LEnNfaGnoebwSzoWjA/MzA8PTek1wjD6Dzx//KXv7Cn/C+//KIxvOL999/XveS+sMHcxcXl4sWLISEhNjY2vUbO2trapKQkd3d3c3NzmUwWERGRnZ3NrXDjxo0lS5bY2trS59sdPXqU3jJNCPnTn/7Erakj9KGP0HNHDUcfQTBDOwd6TKAMdV4M5fqc0S9usEHY2tp6zpw5Z8+enTdvnrW1NbvkoqKihISE3/3ud9bW1uPHj589e/bevXu5sy8jAnNbYuQIzOC6VAt6pRFN+/ssYDihISsrKy4ujhmrE4DPmDGjtra2oqKC74bo5R//+MfOnTsLCgqMs7ox/t3QRh8Mpn23FYwpBjwv8I3Sk5FDHxf6CB0EAkFmZibuHKbQYwJlqPOCl2/Uk08+2dbWdvfuXWOuVAdEYN1wFaEBvdKIpv19Nti88WBkX3zxhTHnPYLB2bdvn+A32mPd7969u2jRoqamptraWrbazJkz29vbudW47woEAl9fXyNuwcAcO3bM09NTx/jGoqKiyMhIOzs7qVQ6f/58jVs6jVknOTlZe1b/5ORkdj/Pnj1br20G40Lo0xN2lOlDB6EBHQRfVCrV+PHjubcK3rlzp7S0lHvrBwwIIvCIgCCswZSDJ3IWI0l6evrSpUtbWlq++OKLhw8fInc4Unz++ecMw7S0tHALi4qKfH19w8PDbW1tHRwcGIahM0sVFRUlJSVxa9J38/Pz6TPGefmFuV+lpaWLFi1KSUmhz4Po1YULF+bMmSOVSq9fv15WVubh4TFv3ryffvqJlzqrVq1KSUl5++23uZ9iH6JuZmY2pN0BBoXQpyfsqJEIHQSFDoJfDx8+TEhIKC8vf/To0cWLF+Pi4mxtbTX2AOiGCDxCIQhTph48uTeKjNk7MPuaYsPU7N27lxAiEommT59eWFhozFUb+bshkUjmzp1rysvX877Bb775hvwWDbkaGxtdXV0TEhK4hUql0tLS0t7enhCyf/9+jY+w0dA0LV++fOvWrV1dXS4uLmZmZtoVuru7vb295XL5o0ePaIlarfby8nJzc2tvbzd+HYZhioqK6NBB7daamZn5+/v3u9W8zGdhTDr6jnfffddozeAx9FHoI/pFjHjnsOl3EHpGBnQQrFHZQTB8zGcxaP/617+WLl06efJkCwsLJyenFStW/Pvf/x7WNeoPEVgfRr6KMP04rM/ZhyDMMrXgqf19xjgLQghZt24dd6ds3ryZ7xb1Lj4+nmGYrq6u4uLip59+mu/mwOBt27ZNpVK98847GuVWVlYZGRlCoTAhIUHjMWMm7ssvv0xOTtYx3uzcuXO//vprTEyMWCymJWZmZsuXLy8vLz969Kjx6xBCFApFTEzMm2++iQeM9UVHd/Lf//3fRmsG76EPfQQYEzoIgg7C6MLCwv73f/+3rKyso6NDpVLt27dvypQpfDfq/0MEBiNDECamFzyRswAwNoZh0tPT/f39J06cqP1uRETEpk2bmpubY2NjNW6ZM2Vs/OrLqVOnCCEat/nRl9nZ2cavQy1durSiosLIT5gHAOgLOggWOggAMD4EYZZJBU/kLMDA6DOcpkyZYmFhMW7cuIULF54+fZq+tXnzZjr/SmBgIC05ceIELXFwcKAlqampAoGgtbU1NzeXvkWTgrRcIBC4uroqlcqwsDCpVGptbR0SEsLO/jKU5RtTcXFxdXW1QqHoq8K7774bHh5+5cqV119/XfeidOztw4cPsxPe3LlzJy4uzs7Ozt7ePioqqrS0lLuQmpqaxMREOiLU0dExOjq6qKho6JupoaSkhBDi6urKLXRxcSGEsLlqY9ahZsyYQQj58ccfh7BlAKAvdBD9QgfBQgcBMBwQh3VDEGaZVPBEzgIMSaVS+fn57d+/Py0trba29sKFC9bW1mFhYenp6YSQTZs2Mf/5EOwFCxYwDMM+PZv8NgKQexsbHVlEyxUKRUNDw5o1azZv3qxSqc6dO1dfXx8aGnr27NkhLp8KDQ21t7c/f/78cO4kcvXqVaJ1znMJhcKMjAw3N7f09PSMjIy+qune20uWLGEYZvHixYSQpKSkpKSk+/fvZ2Zmnjp1avny5exCqqqq/Pz8srKydu3aVV9ff+bMmfr6+oCAAO2HhA9RQ0MDIUTjEeh0luaHDx8avw5FQy09IgAwrNBB6AMdBAsdBIDBIQ73C0GYZVLBEzkLMKSUlJSysrIdO3ZERUXZ2tp6enru379fLpcnJibqmKh2QFpbW3ft2hUQECCRSHx9ffft29fZ2blmzRqDLLynp4fGR4MsrS9VVVWEEJlMpqOOg4NDVlaWubl5QkICzWtq039vx8fH0z02f/78yMhIpVJZW1vLLuTu3bvbt29/5plnbGxsvL29Dxw4wDBMv8ljg6C7WiAQ8FXH1tZWIBDQIwIAwwodhD7QQbDQQQAYHOJwvxCEWSYVPJGzAEM6dOgQISQyMpItsbS0DAsLa2trM9QAIYlEQkccUdOmTZs4cWJxcbFBTgw2fzn0RelA738zNzfXXW327Nmpqamtra2xsbFtbW3aFfTf235+fuzfbm5uhJDKykr68vDhw0KhMBZCFUkAACAASURBVCoqiq3g7Ozs7e1dWFhYUVEx0E3Twc7OjhDS2trKLaQv6VtGrsMSiUS97l4AMCx0EPpAB8FCBwFgcIjD/UIQZplU8ETOAgymo6OjsbHRyspKKpVyy52cnAghKpXKIGvRPismTJhACHnw4IFBlm8EVlZWhJCurq5+ayYmJsbFxV29evW1117TeGtAe5ubLbawsCCE9PT0sAvp6emRyWQCjkuXLhFCbt26NbgN7NWTTz5JCNGIsPfv3yeEeHp6Gr8OS61W9zs1EQAMEToIPaGDYKGDADAsxGF9IAizTCp4ImcBBmNpaSmTydrb25ubm7nldASUs7MzfSkUCjs7O7kV6P1RXDqGIdXV1WkMCaNBkAbEoS/fCORyOSGksbFRn8rp6eleXl5fffUVfYg0S8+9rZulpaWdnZ1IJOrq6tJ+mGVISIi+m6QHurTCwkJuIX0ZFhZm/DpUU1MTwzD0iADA8EEHoSd0ECx0EACGhTisDwRhlkkFT+QswJCWLl1KCOE+26ajoyM7O1ssFkdERNASuVxOc3KUSqW6d++exnKsra3ZcObl5bVnzx72rfb2dqVSyb785ZdfKisrFQoFe2IMcflG8NRTTxGtPGVfbGxsvv/+e4lEsmvXLo239Nnb/YqOjlar1eyUztRHH300adIkwz5XOTg4eOrUqQcPHmQfDdXd3X3gwAE3Nzd24Jwx61D0q0KPCAAMK3QQ+kAHQUvQQQAMB8ThfiEI0xKTC57chE1mZqZGCQCl53ejqqrK3d3dycnpyJEjTU1NN27ciI6OFggEe/bsYevQAVR///vfm5ub//3vfy9btszFxcXe3p67nAULFshksnv37uXl5YlEomvXrtFyhUIhk8nCwsLy8vJaWlqUSuX06dMtLCzOnDljkOWHhISMHz8+Pz+/3y2NiYmJiYnptxpNu37++efcwp6engkTJrDzIbOUSqVMJut1Ofv27SOEaGyFPnubzkjc1tbGlqxfv54QcvnyZfqyurp6ypQpHh4ex44da2hoqKur++KLL6ytrTMzM9mPrFixghBy+/btfreXYRgXFxczM7Ne38rPz7eysnruueeqqqpqa2sTEhJEItGJEyf4qsMwzP79+wkhhw4d0ig3MzPz9/fvd2MNGDP1/EYBmCZCCDdo9GrsdBB6RgZ0EFyjr4Ng9Dsv9IHrc+iXnlcRYycO63P2IQhzmVTw1P4+I2cBetH/u1FbW5uUlOTu7m5ubi6TySIiIrKzs7kVGhoa4uPj5XK5WCwODAxUKpXsI47Wr19P65SUlAQFBUkkEjc3t507d7KfVSgULi4u165di4iIkEqlYrE4ODg4JyfHUMsPCgoaN25cXl5ev5s5lJwFwzAbNmwQiUT379+nL2tqariZRB8fH+1FrV69WiMaMjr3tsZjkDZu3Mj851C9yMhIWpM+PtrDw8Pc3NzR0TE8PPzkyZPctYSGhtrY2KjVah1beuTIEe2U6N69ezWqXbp0aeHChba2tjY2NqGhoRrHzvh1YmNjXVxcOjs7NcqRswAYED3/NxsjHcRQchYMOojR0kEwyFmAEel/FTFG4rA+Zx+CsEY10wmeyFnAIJnId4OGQr5bwTBDzlk0NDS4uLgkJCQMT+sM6eHDh2KxOD4+nu+GGF5RUZFAIPj222+130LOAmBADPW/2VCYTgcxxJwFOghTMPQOgkHOAozIRK4iTCcODyVngSA8FAYJntrfZ8xnAcADmUx25MiRgwcP7ty5k++26MIwTGJioq2t7fvvv893Wwzs9u3b0dHRKSkpzz33HN9tAQD4P+ggeIcOAmAsQxAetOELnshZAAy71atXCwQCGxsbbuHMmTMLCgqOHz/e1NTEV8P6VV1dffv27ezsbD2nOB5Bdu/evWXLli1btnALk5OT6XOkuru7+WoYAIwp6CBMEDoIgLEDQdiAhi94ImcBI0NqaqpAICguLr5//75AINi0aRPfLdLLypUr2UFNLS0tGu9Onjz56NGjtra2vLRNH87Ozjk5Od7e3nw3xPA++ugj7Rzwhx9+yB6v8+fP89IwABgodBC8QAcBAKyRGIcRhA1u+IKnaMhtAzCGdevWrVu3ju9WAACAyUEHAQDAL8RhGFYYZwEAAAAAAAAApgg5CwAAAAAAAAAwRchZAAAAAAAAAIApQs4CAAAAAAAAAExRL3NwZmVlGb8dYOLy8/MJvhscFRUVBDuEEEJId3d3eXn5pEmThMIxlwOl54WhVFRU4BsFI5dhT4cRDT0mDIfR/Y3q6em5d+/eY489JhAI+G7LiITrUm3olUauiooKV1fX/yhiODIzM3lqGADASMUYQkxMDN/bAQAABpaZmTn0DgLX5wAw1sTExHDDoIBhGL6bBAAjWHd3d0lJSWFhYW5ubk5OzvXr1xmGkcvlgYGBc+fO9fHx8ff3Nzc357uZAGASsrKy4uLicO0BMDZVVlbSq4Xc3NzLly/39PSwFwyBgYEzZ84cg2M2AaBfyFkAgCE1NTVdvHgxJyeHZjEePnxoY2OjUCh8fHwCAwNDQkIcHBz4biMA8AY5C4AxRa1WFxcX06uCs2fP3rt3TyQSKRQKmqQIDg6eMGEC320EAFOHnAUADJdeh2B4eHjQ8Rf4RQVgDELOAmDUq66uvnjxItv7t7e3Ozk5+fn50a4/MDDQysqK7zYCwEiCnAUAGEljY6NSqaQ/tuTk5DQ0NNAhGHRQ6Jw5c+zt7fluIwAML+QsAEal27dv0zs+6E8UQqHQy8uLJinmzp07depUTK4JAIOGnAUA8ABDMADGJuQsAEaH1tbWy5cv0048Ly+vvr6e+zvE3Llzx48fz3cbAWCUQM4CAPjHDsHIzc3Ny8t79OiRVCqdPn06Ln0ARhnkLABGLnYGzcLCQqVS2dnZiSm3AcAIkLMAANOiVqtv3LjBXhVhCAbAaIKcBcAIwu2Rf/755zt37ohEIk9PT5qn+P3vfz958mS+2wgAox9yFgBg0lQqlVKppHeR5ObmtrW12drazpo1i01hjBs3ju82AoC+kLMAMHHs87/YkY9st0tTFWKxmO82AsDYgpwFAIwYGkMwrl27ZmZmhlm+AEYQ5CwATBCdQZP+PHDp0iV2eCP6VgAwBchZAMBIVVVVVVBQgCEYACMIchYApuDRo0eXLl2iHejp06dra2utra1nzpxJe8+QkBAHBwe+2wgA8P8hZwEAowF3CEZubu7t27cxBAPABCFnAcAXmuinHWVBQUFHR4dcLmd7ST8/P0tLS77bCADQC+QsAGAUqqysZB+kWlhY2N7eLpPJ/Pz86BCMoKAgOzs7vtsIMBYhZwFgNPSx4to3VLJP+vD29ua7jQAA/UPOAgBGObVaXVxcTK/YcnJyysrKMAQDgC/IWQAMq+bm5uLiYpqnyMnJaWhokEql/v7+SNkDwMiFnAUAjC3cIRh0cKyTk5Ofnx9NYcyZM8fa2prvNgKMWshZABhcZWUle2vk5cuXe3p65HI5TcrjAeEAMAogZwEAY1dXV9eVK1foEIxz587dvXuX++R5jJsFMDjkLACGjh0/mJube/bs2QcPHpibm0+fPp0mKYKDgydMmMB3GwEADAY5CwCA/097CIazs7Ovry97FwkeSg8wRMhZAAxOdXX1xYsX2U6qvb2d20MFBgZaWVnx3UYAgGGBnAUAQC/YIRi5ubnnzp2rrq7GEAyAoUPOAkB/t2/fpt1QTk7O9evXhUIhJmMCgDEIOQsAgP6xdwsXFhYqlcrOzk76iDgMwQAYEOQsAHRoaWkpKiqi3U1eXl59fb2NjY1CoaAdzdy5c8ePH893GwEAjA05CwCAgWltbb18+TIdoEtvJBaJRAqFgp2V3d3dne82Apgo5CwANHBz4hcvXuzq6mJn0PTx8fH39zc3N+e7jQAAfELOAgBgSPoagkGvOH19fXGPMQALOQsAtVp948YN2nH8/PPPd+7c4Sa+g4ODH3vsMb7bCABgQpCzAAAwGO4QjDNnztTU1LBzufv4+Pz+97+fPHky320E4BNyFjA2NTU1Xbx4kU5OkZub29bWZmtrO2vWLPqkD9xgCACgA3IWAADDhR2CkZube/ny5Z6eHu4QDD8/P0tLS77bCGBUyFnA2KExgybDMB4eHmySAjNoAgDoCTkLAABj4M6sdv78+draWu4QDAwGhjECOQsYxR49enTp0iU61O706dO1tbUSiWTGjBk0VR0SEuLg4MB3GwEARh7kLAAAeEB/f6OXthiCAWMHchYwylRVVRUUFNB8dEFBQUdHBzeYz5o1y8LCgu82AgCMbMhZAADwrLm5ubi4mF7y5ufn19XVWVtbz5w5k171BgcHT5gwge82AhgGchYw0nV3d5eUlLBTL1+7ds3MzMzLy4smKQIDAz08PPhuIwDAqIKcBQCAael1CAb73Dv8agcjGnIWMBI1NzdfuHCBRuacnJyGhgapVOrv78/OryyTyfhuIwDAqIWcBQCA6eIOwcjLy6uvr+feHT1v3jxHR0e+2wgwAMhZwEjR6yTK7GCKmTNnCoVCvtsIADAmIGcBADBicGehLykp0RiC4e/vb25uzncbAXRBzgJMVldX15UrV2iMPXv27IMHD9iZkpEjBgDgEXIWAAAjUlNT08WLF9m7SB4+fGhjY6NQKDAEA0wZchZgUqqrqy9evEijaE5OTnt7u7Ozs6+vL80F+/r6WllZ8d1GAICxDjkLAIARj84Jx152X79+nWEYDGMGE4ScBfBLO1oKhUIvLy/2SR9Tp04VCAR8NxMAAP4PchYAAKMNdwgGnS6ODsGgV+QBAQEODg58txHGKOQswPhaWlqKioq4EwNxQ2JgYOC4ceP4biMAAPQJOQsAgNGs1yEYHh4edAoMDMEAI0POAoyDnUGzsLDw4sWLXV1dmP0HAGCEQs4CAGAMaWxsVCqVdJK5vLy8R48eSaXS6dOn00v5OXPm2Nvb891GGM2Qs4Bholarb9y4QfMU586du3v3rkgkUigUNEkRHBz82GOP8d1GAAAYDOQsAADGKDoEg/0pEkMwwAiQswADYu+Dy83Nzc3NbWtrk8lkfn5+9I6PuXPnisVivtsIAABDhZwFAAAQQohKpVIqlfQuEnr1zx2CMXfu3PHjx/PdRhh5KioqXnrppe7ubvry4cOHZWVlTz/9NFvBy8tr9+7dPLUORh7uI5+5mVbMoAkAMFohZwEAAJq4o6wLCwuvXbtmZmaGqfVhcB5//PHS0tK+3n377bffe+89Y7YHRpZHjx5dunSJplNPnz5dW1srkUhmzJiBO9oAAMYI5CwAAKAfVVVVBQUF3CEYtra2s2bNYu8iwaz7oMN77723efPmrq6uXt/99ddfp06dauQmgYmrrKxkZw4uKCjo6OiQy+VsznTWrFkWFhZ8txEAAIwEOQsAABgADMGAgSotLX3iiSd6vd7w9va+evWq8ZsEpoY7vU5ubu7t27dpYKFRJSgoyN3dne82AgAAP5CzAACAwaNDMNgURnt7OzsHno+PT1BQkJ2dHd9tBP7NmDHjypUrGpcc5ubmW7Zseeutt/hqFfCrubn5woULNHTk5OQ0NDRIpVJ/f38aPX7/+9/LZDK+2wgAAPxDzgIAAAxDrVYXFxez/4GUlZVhCAZQ27dvX79+vVqt5hYKBILbt29PnjyZp0YBDyorK9nBFJcvX+7p6cGzigAAQDfkLAAAYFhw70jXGIIRGBg4Z84ca2vrgS6zq6vrxRdf3LZtm5ub23C0GYZJVVWVq6trT08PWyIUCv39/fPy8nhsFejvyJEjP//887Zt2wb6wa6uritXrtAkxZkzZ2pqaszNzadPn07jwLx58xwdHYejwQAAMGogZwEAAMOO/b+lsLDw559/vnPnjkgk8vT0pOMvfHx89ByCUVBQ4OfnJxaLt2zZ8vrrr4tEIiM0Hgzi97//fW5uLpu2EIlEf//73//85z/z2yro1927d1evXn38+HFnZ+eqqip9PkIfnMzNVzo7O/v6+tLz3c/Pz9LScribDQAAowZyFgAAYGzaDwWg/9Kwd5GIxeJeP/j3v/997dq1arVaKBT+7ne/++qrr2bNmmXkxsPg7N27989//jObszAzM6usrJwwYQK/rQId1Gr13/72t40bN3Z3d9PHvpSXl7u6umrXpDNosif19evXhUIh974wb29vozcfAABGCeQsAACAT9yh4+fOnauurtYYgsH9b+e55547ePBgd3c3IUQkEnV3d69YsSItLW38+PH8bQHo5eHDh05OTvRfXzMzs/nz5584cYLvRkGfioqKXnnllStXrrBpJqFQuH///ri4OPqypaWlqKiInZzi4cOHNjY2CoWCnrl4BDIAABgKchYAAGBC2Cn6CgsLlUplZ2enXC738fGhP9i+8MILKpWKW18kEslksu3bt7/44ot8tRn0FBkZ+eOPP3Z3dwuFwq+//nrlypV8twh68ejRo/fee+/jjz8WCoXcaVMtLCxWrFgRGhqal5eXm5v766+/dnd3T5kyZc6cOXPmzKGDKTCDJgAAGBxyFgAAYKJaW1svX75MB5yfPXv2wYMHvVYTCoUMwwQFBe3Zs8fLy8vIjQT9ffvttytWrGAYxtLSsra21sbGhu8WgaajR48mJCQ8ePBA4yEvlEwma21tVSgUdAzUvHnzJk2aZPxGAgDAmIKcBQAAjAw7d+58/fXX++q2RCKRUChMSUlJSUnBDH+mqbW11cHBob29PSYm5rvvvuO7OfAfqqqq1q1bt3//fqFQyH3CC5dQKKyurnZwcDBy2wAAYCzDED4AABgZ7t69a25u3te7arW6s7Nz8+bN06dPz8nJMWbDQE8SiWTJkiWEENwVYlJ6enp27tz5xBNP0ERSXwkL+tbVq1eN2DQAAACMswAYmfR5KiQAAAAAGArGiAHwAk+2BxipkpKSAgIC+G4FgJF0d3e/+OKL9B57MzMzgUDA3m9vZmZmZ2fn4ODg7Ozs6Ohob2/v4OBgb2/v6OhoYWFBCImLi8P5wsrPz9+xY0dmZiYva+/u7s7MzHz++ed5WTtoa25urqurq6urq6mpqa+vr6ure/DgQW1tbWNjI31AD/ntxiu1Wt3T06NQKDZs2MBvmwF48emnn/LdBIAxCuMsAEYkgUCQmZm5bNkyvhsCYCRKpXLhwoVyudzDw8Pd3f2xxx5zc3Nzc3ObNGmSXC7X/VmcL1xZWVlxcXE89v7t7e1WVlZ8rR30xDCMSqUqLy+vqKgoLy+/d+9eeXl5WVlZe3v7lStXMNYPxqDY2FhCCMZZABgfxlkAAMAI4OfnV1tby3crwACQsBgRBAKBXC6Xy+WzZs3iuy0AADCmYQ5OAAAAAAAAADBFyFkAAAAAAAAAgClCzgIAAAAAAAAATBFyFgAAAKBp3759gt/Y2NhovHv37t1FixY1NTXV1tay1WbOnNne3s6txn1XIBD4+voacQsG5tixY56eniJRn/N8FRUVRUZG2tnZSaXS+fPn5+bm8lUnOTl5iI99weHD4TMpRjh8ve725ORkdv/Mnj3bINsCAMMBOQsAAIBetLS0PPHEE1FRUXw3hE+ff/45wzAtLS3cwqKiIl9f3/DwcFtbWwcHB4ZhlEolLU9KSuLWpO/m5+fb29szDFNQUGDU1uuntLR00aJFKSkp1dXVfdW5cOHCnDlzpFLp9evXy8rKPDw85s2b99NPP/FSZ9WqVSkpKW+//fbgtheHD4fPdBjt8PW62z/88EOGYRiGMTMzM/imAYAhMQAwAhFCMjMz+W4FwMgwuPOlqanJw8Nj4cKFw9EkfUgkkrlz5xp8sfTHxn6rffPNN+S3nAVXY2Ojq6trQkICt1CpVFpaWtrb2xNC9u/fr/ER9r8m07R8+fKtW7d2dXW5uLiYmZlpV+ju7vb29pbL5Y8ePaIlarXay8vLzc2tvb3d+HUYhikqKqJP8B3oxuLwMTh8psRoh4/RudvNzMz8/f37bW1MTExMTMxAtxEAhg7jLAAAAHohlUpLS0uPHTvGd0NMy7Zt21Qq1TvvvKNRbmVllZGRIRQKExISbt68yUvbBufLL79MTk7WMS793Llzv/76a0xMjFgspiVmZmbLly8vLy8/evSo8esQQhQKRUxMzJtvvqlWqwe0sTh8BIfPlBjt8JEh7HYA4B1yFgAAAKAXhmHS09P9/f0nTpyo/W5ERMSmTZuam5tjY2M1bq03Zez/OX05deoUIURjOgD6Mjs72/h1qKVLl1ZUVPzwww/9bd//weFj4fCZCKMdPmoQux0ATAFyFgAAAJoOHz7Mzs1G/wHglty5cycuLs7Ozs7e3j4qKqq0tJR+KjU1lVZwdXVVKpVhYWFSqdTa2jokJISdEG7z5s20TmBgIC05ceIELXFwcOAup7W1NTc3l76l43dIYyouLq6urlYoFH1VePfdd8PDw69cufL666/rXlRdXd3atWunTJliYWExbty4hQsXnj59mr6lz66mampqEhMTJ0+ebGFh4ejoGB0dXVRUNPTN1FBSUkIIcXV15Ra6uLgQQtjftI1Zh5oxYwYh5Mcff9R/Q3D4WDh8Y+3wUYPY7QBgCpCzAAAA0LRkyRKGYRYvXtxrSVJSUlJS0v379zMzM0+dOrV8+XJaZ926dQzDKBSKhoaGNWvWbN68WaVSnTt3rr6+PjQ09OzZs4SQTZs2MQwjkUjYJS9YsIBhGB8fH7aELoc7nwV3MHNoaKi9vf358+eHeydou3r1KtH634BLKBRmZGS4ubmlp6dnZGT0VU2lUvn5+e3fvz8tLa22tvbChQvW1tZhYWHp6elEv11NCKmqqvLz88vKytq1a1d9ff2ZM2fq6+sDAgLy8/MNuc2ENDQ0EEK4h4wQQp+l8vDhQ+PXoei/ZPSI6AmHj4XDN9YOHzWI3Q4ApgA5CwAAgIGJj48PCAiQSCTz58+PjIxUKpW1tbXcCq2trbt27aJ1fH199+3b19nZuWbNGoOsvaenhyYyDLK0AamqqiKEyGQyHXUcHByysrLMzc0TEhLo75/aUlJSysrKduzYERUVZWtr6+npuX//frlcnpiYqPH4AB27OiUl5e7du9u3b3/mmWdsbGy8vb0PHDjAMEy/PzIbBN3/AoGArzq2trYCgYAeET3h8LFw+Mbm4RvEbgcAU4CcBQAAwMD4+fmxf7u5uRFCKisruRUkEgkdhExNmzZt4sSJxcXFBrlWZn/SHPqiBoreJmNubq672uzZs1NTU1tbW2NjY9va2rQrHDp0iBASGRnJllhaWoaFhbW1tWkM29axqw8fPiwUCrkPo3V2dvb29i4sLKyoqBjopulgZ2dHCGltbeUW0pf0LSPXYYlEol53b19w+Fg4fGPt8LEGutsBwBQgZwEAADAw3J86LSwsCCE9PT3cCtoXyhMmTCCEPHjwYPhbN4ysrKwIIV1dXf3WTExMjIuLu3r16muvvabxVkdHR2Njo5WVlVQq5ZY7OTkRQlQqFbewr11NF9LT0yOTyQQcly5dIoTcunVrcBvYqyeffJIQovGf2P379wkhnp6exq/DUqvV/U5hyIXDx8LhG2uHjzXQ3Q4ApgA5CwAAAAOrq6vTuHeDZito5oIQIhQKOzs7uRXoLdlcukc+80IulxNCGhsb9amcnp7u5eX11VdfffPNN9xyS0tLmUzW3t7e3NzMLafj0p2dnfVZuKWlpZ2dnUgk6urq0n6Qe0hIiL6bpAe6tMLCQm4hfRkWFmb8OlRTUxPDMPSI6AmHj4XDN9YOHzWI3Q4ApgA5CwAAAANrb29XKpXsy19++aWyslKhULDXynK5nP4MSKlUqnv37mksxNrams1reHl57dmzZ5hb3b+nnnqKaP2e2RcbG5vvv/9eIpHs2rVL462lS5cSQrhPHOzo6MjOzhaLxREREXo2Jjo6Wq1Wsw9koT766KNJkyZxpywduuDg4KlTpx48eJB9hGR3d/eBAwfc3NzYAfbGrEPR7w89InrC4aMlOHxk7B0+ahC7HQBMAXIWAAAABiaTyTZs2JCfn9/a2lpQULBy5UoLC4u0tDS2Qnh4eGVl5WeffdbS0lJaWrpmzRp2CAbr6aefvnnzZnl5eX5+/u3bt4OCgmg5j88NUSgUEyZMKC4u1rO+t7f37t27tcu3bt3q7u6elJR09OjR5ubmmzdvPv/881VVVWlpaXSMuj62bt06ZcqUP/7xj8ePH29sbKyvr9+9e/d7772XmprKPhp25cqVAoGgrKxMz2X2SigUfvnll/X19a+88opKpaqrq/vLX/5y69atvXv30uH6Rq5D0edKhoeHsyX9biwOHw4fGauHj9Le7QAwMmiP6QIA00cIyczM5LsVACPDIM4XOk0da8WKFRrP8Nu4cSPzn3d/REZG0s8qFAoXF5dr165FRERIpVKxWBwcHJyTk8NdfkNDQ3x8vFwuF4vFgYGBSqWSfdbp+vXraZ2SkpKgoCCJROLm5rZz5072s0FBQePGjcvLyxvc3sjMzNSn96dDyj///HON8g0bNohEovv379OXNTU13J3g4+OjvajVq1fb29trFNbW1iYlJbm7u5ubm8tksoiIiOzsbPqW/ru6rq5u7dq1Hh4e5ubmjo6O4eHhJ0+e5K4lNDTUxsZGrVbr2NIjR45oXx3t3btXo9qlS5cWLlxoa2trY2MTGhqqcUCNXyc2NtbFxaWzs3NAG4vDh8PHjOHDp73bKTMzM39/fx3tpGJiYmJiYvqtBgAGh5wFwIhEkLMA0JuRzxeaszDa6gZqiDmLhoYGFxeXhISE4WmdIT18+FAsFsfHx/PdEMMrKioSCATffvstW6LnxuLwmQIcPl5o73YWchYAJg73hgAADDulUvnyyy+7u7uLxeLx48c/9dRT//Vf//X555+Xlpby3TSAgZHJZEeOHDl48ODOnTv5bosuDMMkJiba2tq+//77fLfFwG7fvh0dHZ2SkvLcc8/RU/vhFQAAIABJREFUEv03FoePdzh8vNDe7QAwgiBnAQAwjHp6et566605c+ZMmDDh+PHjDQ0N169f//TTT5uaml599dXHH3/csNOVARjW6tWrBQKBjY0Nt3DmzJkFBQXHjx9vamriq2H9qq6uvn37dnZ2tp6PQhhBdu/evWXLli1btrAlA9pYHD5+4fDxQnu3E0KSk5PpU1q7u7v5ahgA6EPA/OddagAwIggEgszMzGXLlvHdEMOwsbGZMWNGTk7O6Fv7xo0bP/jggz179qxatYpb3t3d/eyzzx4/fryrq4udscxEjL7DYbTzJTU19a233mJfbty4cfPmzcO90oHKysqKi4tD7w8AMCCxsbGEkO+++47vhgCMORhnAQAwXEpKSj788EMfHx+NhAUhxMzM7O233+alVTB81q1bx7390gQTFgAAAAAji2n9uAcAMJrs2bOnp6eH/jKjLSAgAL91AwAAAADogHEWAKMZfRTZlClTLC0tXV1d58+f/z//8z9tbW3aFSwsLMaNG7dw4cLTp0/Ttw4fPiz4zZ07d+Li4uzs7Ozt7aOiojRmjtS9FrVanZmZ+Yc//MHZ2VksFk+bNi0tLa2np4e+m5qaKhAIWltbc3Nz6bq4N0rU1NQkJiZOnjzZwsLC0dExOjqaPlxd/+YN09r1dO7cOULI9OnT9ayPwzGshwMAAAAARh5jP6gEAAyB6PHsxqqqKnd3d2dn5yNHjjQ1NalUKjqD96effsqt4OTkdOTIkcbGxhs3bkRHRwsEAu6j0RcvXkwIWbx4cV5eXktLy8mTJ8VisZ+fn/5roU9f/+CDD+rr62tqav72t78JhUKNIfQSiWTu3Lka7a+srHzsscecnJx++OGH5ubmq1evBgcHW1lZ5eXl6d+8YV17SEjI+PHj8/Pz+zoEcrmcEHLhwoW+KnDhcAxx7Troc76MHXo+6xQAALjwrFMAvuCqBWBE0ud/sJdfflm72oIFC9h/X2kF7rPK29vbJ06cKBaLVSoVLaH/hR45coStExMTQwipqanRcy1HjhyZN28e992VK1eam5s3NjayJb3+m/rSSy8RQjIyMtiSqqoqS0tLHx8ftqTf5g3r2oODg8eNG6fj32aas7h48WJfFbhwOIa4dh2Qs+BCzgIAYBCQswDgC+azABi1Dh06RAhZuHAht/D48eMaFSIjI9kSS0vLsLCwb7755scff3zxxRfZcj8/P/ZvNzc3QkhlZaWDg4M+a4mKioqKiuK+q1Ao9u3b9+uvvwYEBOho/+HDh4VCIfezzs7O3t7ehYWFFRUVrq6u+jRvWNd+5swZHUsghEycOLGqqqq2tlZ3NQqHwyBr70t+fn6/dcYIuiuysrL4bggAwEiiZ3cDAAaHnAXA6NTR0dHY2GhlZSWVSgdUwcnJiRCiUqm4hTKZjP3bwsKCEELnIOh3LYSQxsbGTz755NChQxUVFQ0NDWz5o0eP+m2/xqpZt27d4l439NU846xdh+Dg4MLCwitXrmgkEfpaIw7H0Nfelx07duzYsaPfamNHXFwc300AABhh6OBBADAyzMEJMDpZWlrKZLL29vbm5uYBVaiuriaEODs7G2QthJBnn332/fffX7Vq1c2bN3t6ehiG+fTTTwkhDOeRGQKBQHvJdnZ2IpGoq6tLe4RYSEiIPs3jfe0JCQkikejgwYO9vvvXv/5VKBSWlJQQHI7hXzvuDWHh3hAAgEFAwgKAL8hZAIxaS5cuJYQcO3aMWzhz5sw33niDW+GHH35g3+3o6MjOzhaLxREREQZZS3d3d25urrOzc2JioqOjI/13lPvgEsra2rqzs5P+7eXltWfPHkJIdHS0Wq3Ozc3l1vzoo48mTZqkVqv1aRu/ayeEeHp6vvvuuwUFBV999ZXGWzdu3Ni9e/eyZcuefPJJWoLDMaxrBwAAAIARie+UJQAMBtH7uSFyufzo0aNNTU3l5eWrV692cnK6e/cutwJ9UEVTUxP7oIo9e/awC6GzKra1tbEl69evJ4RcvnxZz7WEhoYSQrZt21ZTU/Po0aNTp05NmjSJEHLy5El2mQsWLJDJZPfu3cvLyxOJRNeuXWMYprq6esqUKR4eHseOHWtoaKirq/viiy+sra25G95v84Z17f0+N4RKTk42Nzdfv379jRs3Ojo6Kioq0tPT5XJ5YGBgS0uLxvHC4Rj02nXQ53wZOzDOAgBgEDAHJwBfcNUCMCLp+T9YbW1tUlKSu7u7ubm5XC5/7rnnbt682VcFmUwWERGRnZ1N39KYs3Djxo0MZwA/ISQyMlKftdTU1CQkJLi5uZmbmzs5Ob388svJycl0CexDH0pKSoKCgiQSiZub286dO9nP1tXVrV271sPDw9zc3NHRMTw8nP3nVs/mDdPaqaCgIN3PDWFdvHjxhRdeoM2QSqWzZ89OS0vr6OjQcbxwOAZ6OHRAzoILOQsAgEFAzgKALwLmPy8rAWBEEAgEmZmZy5Yt47shACMAzheurKysuLg49P4AAAMSGxtLCPnuu+/4bgjAmIP5LAAAAAAAAADAFCFnAQAAADBK3L17d9GiRU1NTbW1tYLfzJw5s729nVuN+65AIPD19eWrwX15+PDhF198ERoaOn78eLFY/MQTT6xYsaK4uFi7ZlFRUWRkpJ2dnVQqnT9/vsZkvaxjx455enqKRKK+1qjPcnTXSU5OpvdeAQCAASFnAQAAADAaFBUV+fr6hoeH29raOjg4MAyjVCppeVJSErcmfTc/P9/e3p5hmIKCAp6a3Ke33nrr9ddfX7x48bVr1+rq6r766quioiIfH5/Dhw9zq124cGHOnDlSqfT69etlZWUeHh7z5s376aefuHVKS0sXLVqUkpJCnx7dK32W02+dVatWpaSkvP322wbaBwAAQAghmM8CYETC/fkA+jPm+WJjYzNjxoycnByTXT7msxitmpqavL29IyMjv/jiC7awoKAgMDDQxsamrq5u//79y5cv537k/PnzUVFRtbW1Rm9s/+Lj483MzHbv3s2WFBcXz5gx44knnrh58yYt6enpmT59en19fWlpqVgsJoR0d3d7e3s/evTo1q1blpaWtNrzzz8/ffr0devWTZ48WaVSaT8gWZ/l6Lmu4uLimTNnHjhwAB306IP5LAD4gnEWAAAAACPetm3bVCrVO++8o1FuZWWVkZEhFAoTEhLY//ZNX3p6OjdhQQhRKBRisbi0tJTNuJ07d+7XX3+NiYmhSQRCiJmZ2fLly8vLy48ePcp+8Msvv0xOTtZxV4g+y9FzXQqFIiYm5s0339TOjAAAwOAgZwEAAAAwsjEMk56e7u/vP3HiRO13IyIiNm3a1NzcHBsbqzGxxQjS2tra1tb21FNPCQQCWnLq1ClCiMZkHPRldnY2W8JmGfqiz3L0XBchZOnSpRUVFT/88IPeWwYAALogZwEAAEAIIXV1dWvXrp0yZYqFhcW4ceMWLlx4+vRp+tbmzZvpVIWBgYG05MSJE7TEwcGBlqSmpgoEgtbW1tzcXPoW/V2XlgsEAldXV6VSGRYWJpVKra2tQ0JC2An8hrJ8AEJIcXFxdXW1QqHoq8K7774bHh5+5cqV119/XfeidJwIhw8fZqftvHPnTlxcnJ2dnb29fVRUVGlpKXchNTU1iYmJkydPtrCwcHR0jI6OLioqGuI20jH5GzduZEtKSkoIIa6urtxqLi4uhJABjSjRZzn6r2vGjBmEkB9//FH/BgAAgA7IWQAAABCVSuXn57d///60tLTa2toLFy5YW1uHhYWlp6cTQjZt2sQwjEQiYesvWLCAYRgfHx+2ZN26dbTO3LlzGYZhGIYODqflCoWioaFhzZo1mzdvVqlU586dq6+vDw0NPXv27BCXT4WGhtrb258/f344dxKYrqtXrxKt/6i5hEJhRkaGm5tbenp6RkZGX9V0nwhLlixhGGbx4sWEkKSkpKSkpPv372dmZp46dYo7U0ZVVZWfn19WVtauXbvq6+vPnDlTX18fEBCQn58/6A2srq5OTk6Oj4/nzhPR0NBACOGeOIQQGxsbQsjDhw/1X7g+y9F/XTSRQY8IAAAMHXIWAAAAJCUlpaysbMeOHVFRUba2tp6envv375fL5YmJiTqeNTAgra2tu3btCggIkEgkvr6++/bt6+zsXLNmjUEW3tPTQxMZBlkajDhVVVWEEJlMpqOOg4NDVlaWubl5QkICHTWgTf8TIT4+nn6Z58+fHxkZqVQq2bk8U1JS7t69u3379meeecbGxsbb2/vAgQMMw/Q7xKMvdXV1CxYsmDdvHnd60b7Qs4C9f2TQ9FlOr3VsbW0FAgE9IgAAMHTIWQAAAJBDhw4RQiIjI9kSS0vLsLCwtrY2Q43xlkgkdNA4NW3atIkTJxYXFxvkfxv2p+yhLwpGIjpLhbm5ue5qs2fPTk1NbW1tjY2NbWtr066g/4ng5+fH/u3m5kYIqayspC8PHz4sFAqjoqLYCs7Ozt7e3oWFhRUVFQPdtNbW1oiIiKlTp2ZkZJiZmXHfsrOzoxU06rNv6Umf5QxoXSKRqNfdCwAAg4CcBQAAjHUdHR2NjY1WVlZSqZRb7uTkRAhRqVQGWYv2PzYTJkwghDx48MAgy4exzMrKihDS1dXVb83ExMS4uLirV6++9tprGm8N6ETgjumwsLAghPT09LAL6enpkclkAo5Lly4RQm7dujWg7VKr1bGxsS4uLl9//bVGwoIQ8uSTTxJCNPIg9+/fJ4R4enrqvxZ9ljOgdanV6n4n/gQAAD0hZwEAAGOdpaWlTCZrb29vbm7mltPB8M7OzvSlUCjs7OzkVqC3uHPpGEleV1ence8GzVbQzMXQl///2LvzuKiq/3/g57LMMAzDgCCrmIiKhTYqkmISCQKaKEosri2k8SkNySXBJXcRf7RQyieEyCxMlh6aaFqES4pgYIErgqAmqyyxKss49/fH/X7uZz4gMKx3Bl7Pv5xz33PO+95xDjPvufceGMxMTU0JITU1NYoER0dHW1tbx8TEfPfdd/LtCr4ROsbn8/X09DQ0NFpaWug2ZsyYoeguEUII8ff3b2pqio+PZ+84O2rUKPa+LUxvV69elX8K89DZ2VnxURTpR/GxamtraZpmXhEAAOg51CwAAADIggULCCHyyxM2NTWlpKQIBAI3NzemxdTUlPlZlVFaWvr333+36kdbW5utO1hbWx88eJDd1NjYmJGRwT68fv16cXGxRCJhv9v0sH8YzMaNG0fanAXQHh0dnR9//FEoFEZERLTapMgboVOenp5SqZRdFocRGho6fPhw+RvHdmrbtm03b9786aef+Hz+MwMcHR1feOGFxMREdgHXp0+fHj161MLCQv7ylk4p0o/iYzHvYuYVAQCAnkPNAgAAgISEhFhaWgYGBp48ebKuri43N3fx4sUlJSXh4eHMifGEEFdX1+Li4v3799fX1+fn569evZo9RYI1adKk3Nzchw8fpqWlFRQUODg4sJvEYvHGjRvT0tIaGhoyMzOXLl3K4/HCw8PZgJ70j3VDBjmJRGJkZJSdna1gvI2NTWRkZNt2Rd4InQoJCbGysvLz8zt9+nRNTU1VVVVkZOSOHTvCwsLY0yWWLl1KUdS9e/fa6+TQoUPbt2+/cuWKSCSSv8ZEflFVNTW1r7/+uqqq6u233y4tLa2srFy5cmVeXl5UVBRzsYyCFOlH8bGYVV1dXV0VTwAAADrS9rQ9AFB+hJC4uDiuswBQDQq+XyoqKgIDAy0tLTU1NcVisZubW0pKinxAdXX18uXLTU1NBQLB9OnTMzIy2LVIN2zYwMTk5OQ4ODgIhUILC4sDBw6wz5VIJObm5rdu3XJzcxOJRAKBwNHR8dKlS73Vv4ODg76+/uXLlzvdzbi4OPz1H5A2btyooaFRVFTEPCwvL5f/vGdra9v2Ke+9956BgUGrxg7eCK0WK920aRP9v5c7zZkzh4msrKxcs2bNyJEjNTU1hw4d6urqmpycLD+Kk5OTjo6OVCptb3c6OFEiLS1NPvLPP/+cPXu2rq6ujo6Ok5NTq7cVTdNJSUltO4mKimoV1mk/CsYwN+Bobm5ub9dARXl5eXl5eXGdBcBgRNFYFw1ABVEUFRcXJ79MPQC0RxneLxMmTKioqOjGogm9Lj4+3tfXF3/9B56amhobGxt3d3dFFgTlVnV1tZmZ2ZIlS6KiorjOpZdlZ2dPnDjxyJEjCxcu5DoX6GXe3t6EkISEBK4TARh0cG0IAAAAgMoTi8VJSUmJiYkHDhzgOpeO0DQdEBCgq6u7c+dOrnPpZQUFBZ6ensHBwShYAAD0ItQsAAAAAAaCiRMnZmZmnj59ura2lutc2lVWVlZQUJCSkqLgQiQqJDIycvfu3bt37+Y6EQCAAQU1CwAAgD4UFhZGUVR2dnZRURFFUZs3b+Y6IxjIRowYcfLkSV1dXa4TaZeJicmlS5dsbGy4TqT3hYaG4gwLAIBep8F1AgAAAAPZunXr1q1bx3UWAAAAACoJ51kAAAAAAAAAgDJCzQIAAAAAAAAAlBFqFgAAAAAAAACgjFCzAAAAAAAAAABlRNE0zXUOANBlFEVNnTp12LBhXCcCoAISExPxfmEVFhamp6d7eXlxnQgAgCpJT0+fOnVqQkIC14kADDqoWQCoJG9vb65TAADostLS0r/++mv27NlcJwIA0GX29vZr1qzhOguAQQc1CwAAAOgn8fHxvr6++OwBAAAACsL9LAAAAAAAAABAGaFmAQAAAAAAAADKCDULAAAAAAAAAFBGqFkAAAAAAAAAgDJCzQIAAAAAAAAAlBFqFgAAAAAAAACgjFCzAAAAAAAAAABlhJoFAAAAAAAAACgj1CwAAAAAAAAAQBmhZgEAAAAAAAAAygg1CwAAAAAAAABQRqhZAAAAAAAAAIAyQs0CAAAAAAAAAJQRahYAAAAAAAAAoIxQswAAAAAAAAAAZYSaBQAAAAAAAAAoI9QsAAAAAAAAAEAZoWYBAAAAAAAAAMoINQsAAAAAAAAAUEaoWQAAAAAAAACAMkLNAgAAAAAAAACUEWoWAAAAAAAAAKCMULMAAAAAAAAAAGWEmgUAAAAAAAAAKCPULAAAAAAAAABAGaFmAQAAAAAAAADKCDULAAAAAAAAAFBGqFkAAAAAAAAAgDJCzQIAAAAAAAAAlBFqFgAAAAAAAACgjFCzAAAAAAAAAABlhJoFAAAAAAAAACgjDa4TAAAAgAGrpaWlvr6efdjQ0EAI+eeff9gWiqL09PQ4yAwAAABUAUXTNNc5AAAAwMBUVlZmbm7+9OnT9gJmzJhx9uzZ/kwJAAAAVAiuDQEAAIC+Ymxs/Morr6ipPfvzBkVRixYt6ueUAAAAQIWgZgEAAAB9aNmyZe1tUldX9/T07M9kAAAAQLWgZgEAAAB96PXXX9fQeMb9s9TV1WfNmmVgYND/KQEAAICqQM0CAAAA+pCuru7s2bPbli1oml66dCknKQEAAICqQM0CAAAA+tbSpUvb3oaTx+O5u7tzkg8AAACoCtQsAAAAoG+5u7tra2vLt2hqai5YsEAoFHKVEgAAAKgE1CwAAACgb2lpaXl6empqarItLS0tS5Ys4TAlAAAAUAmoWQAAAECfW7x4cUtLC/tQV1fXxcWFw3wAAABAJaBmAQAAAH1u5syZQ4YMYf6tqam5aNEiHo/HbUoAAACg/FCzAAAAgD6noaGxaNEi5vKQlpaWxYsXc50RAAAAqACKpmmucwAAAICBLzU1dfr06YQQY2Pj4uJiNTX8cAIAAACdwMcFAAAA6A/Tpk0zNzcnhLzxxhsoWAAAAIAiNLhOAAAA+kNhYeHly5e5zgIGOzs7u6KiIgMDg/j4eK5zgcHOx8eH6xQAAKBzuDYEAGBQiI+P9/X15ToLAABlgc/AAAAqAedZAAAMIviMDh3w9vYmhCQkJPTpKImJiV5eXn06RG+hKCouLg6/xg88qOECAKgQXE0KAAAA/UdVChYAAACgDFCzAAAAAAAAAABlhJoFAAAAAAAAACgj1CwAAAAAAAAAQBmhZgEAAAAAAAAAygg1CwAAAICeevDgwbx582praysqKqj/mDhxYmNjo3yY/FaKoiZPnsxVwu35559/vvrqKycnpyFDhggEgtGjRy9ZsiQ7O7ttZFZW1pw5c/T09EQi0cyZM1NTU5/Z4c8//zxmzBgNjXbXqlOkn45jgoKC4uLiurijAACgGlCzAAAAgB6pr68fPXq0u7s714lwJisra/Lkya6urrq6uoaGhjRNZ2RkMO2BgYHykczWtLQ0AwMDmqYzMzM5Srld69ev/+CDDzw8PG7dulVZWRkTE5OVlWVra3v8+HH5sCtXrkybNk0kEt2+ffvevXsjR4589dVXf/31V/mY/Pz8efPmBQcHl5WVtTecIv10GrNixYrg4OAtW7b00jEAAAAlgpoFAAAA9AhN0zKZTCaTcZWAjo7O9OnTuRq9trZ27ty5r7/++qpVq+Tb+Xy+gYFBZGTkDz/8wFVu3ePn57d69WoTExNtbW0HB4cjR448ffr0o48+YgNkMtk777yjp6f3zTffmJqaGhoa/vvf/7ayslq+fHlTUxMbtmXLlmnTpl29elUkEj1zIEX6USTGysrq2LFju3fvjo+P77OjAgAA3EDNAgAAAHpEJBLl5+f//PPPXCfCjX379pWWln788cet2rW0tGJjY9XU1Pz9/XNzcznJrRuio6MjIyPlWyQSiUAgyM/Pp2maafn9999v3rzp5eUlEAiYFnV19UWLFj18+PDkyZPsE7/++uugoKAOrgpRpB8Fx5JIJF5eXmvXrpVKpT09BAAAoExQswAAAADoJpqmo6Ojp0yZYmZm1narm5vb5s2b6+rqvL29W93YQoU0NDQ8efJk3LhxFEUxLWfPniWEtLoZB/MwJSWFbWGrDO1RpB8FxyKELFiwoLCw8NSpUwrvGQAAqADULAAAAKD7jh8/zt5RkvlaLt9y//59X19fPT09AwMDd3f3/Px85llhYWFMwLBhwzIyMpydnUUikba29owZM9jbK+7atYuJYa/7OHPmDNNiaGgo309DQ0NqaiqzqYNf9ftCdnZ2WVmZRCJpL2Dr1q2urq7Xrl374IMPOu6qsrJyzZo1VlZWPB5PX19/9uzZ586dYzYpckgZ5eXlAQEBI0aM4PF4Q4cO9fT0zMrK6uE+JiQkEEI2bdrEtuTk5BBChg0bJh9mbm5OCOnSGSWK9KP4WBMmTCCE/PLLL4onAAAAyg81CwAAAOi++fPn0zTt4eHxzJbAwMDAwMCioqK4uLizZ88uWrSIiVm3bh1N0xKJpLq6evXq1bt27SotLf3999+rqqqcnJwuXLhACNm8eTNN00KhkO151qxZNE3b2tqyLUw/QqHw5Zdfpmmapmn5SwOcnJwMDAzS09P7bvdv3LhB2nyjlqemphYbG2thYREdHR0bG9teWGlpqZ2d3ZEjR8LDwysqKq5cuaKtre3s7BwdHU0UO6SEkJKSEjs7u/j4+IiIiKqqqvPnz1dVVdnb26elpXV7B8vKyoKCgpYvX+7j48M2VldXE0LkXxpCiI6ODiHkn3/+UbxzRfpRfCymkMG8IgAAMGCgZgEAAAB9Zfny5fb29kKhcObMmXPmzMnIyKioqJAPaGhoiIiIYGImT578/fffNzc3r169uldGl8lkTCGjV3p7ppKSEkKIWCzuIMbQ0DA+Pl5TU9Pf3585a6Ct4ODge/fuff755+7u7rq6umPGjDly5IipqWlAQECrRTc6OKTBwcEPHjz49NNPX3vtNR0dHRsbm6NHj9I03ekpHu2prKycNWvWq6+++tVXX3UazBxn9vqRblOkn2fG6OrqUhTFvCIAADBgoGYBAAAAfcXOzo79t4WFBSGkuLhYPkAoFDKn9DPGjx9vZmaWnZ3dK9882RMNet5Ve5jLYTQ1NTsOmzp1alhYWENDg7e395MnT9oGHDt2jBAyZ84ctoXP5zs7Oz958qTVxQ4dHNLjx4+rqanJLzprYmJiY2Nz9erVwsLCru5aQ0ODm5vbCy+8EBsbq66uLr9JT0+PCWgVz25SkCL9dGksDQ2NZx5eAABQXahZAAAAQF+RPwGBx+MRQlotidr2a6eRkREh5NGjR32fXS/Q0tIihLS0tHQaGRAQ4Ovre+PGjVZLohJCmpqaampqtLS0Wq0JamxsTAgpLS2Vb2zvkDKdyGQysVhMyfnzzz8JIXl5eV3aL6lU6u3tbW5u/u2337YqWBBCxo4dSwhpVQcpKioihIwZM0bxURTpp0tjSaXSTm/8CQAAqgU1CwAAAOBMZWVlq2s3mGoFU7kghKipqTU3N8sHMDc4kNfz6xG6zdTUlBBSU1OjSHB0dLS1tXVMTMx3330n387n88VicWNjY11dnXw7c1WIiYmJIp3z+Xw9PT0NDY2Wlha6jRkzZii6S4QQQvz9/ZuamuLj49l7mo4aNYq9MwjT29WrV+Wfwjx0dnZWfBRF+lF8rNraWpqmmVcEAAAGDNQsAAAAgDONjY0ZGRnsw+vXrxcXF0skEvabp6mpKfOjOqO0tPTvv/9u1Ym2tjZb17C2tj548GAfZ/1f48aNI23OAmiPjo7Ojz/+KBQKIyIiWm1asGABIUR+nc6mpqaUlBSBQODm5qZgMp6enlKplF14hREaGjp8+HD5W5N2atu2bTdv3vzpp5/4fP4zAxwdHV944YXExER2AdenT58ePXrUwsJC/vKWTinSj+JjMf9PmFcEAAAGDNQsAAAAgDNisXjjxo1paWkNDQ2ZmZlLly7l8Xjh4eFsgKura3Fx8f79++vr6/Pz81evXs2egsGaNGlSbm7uw4cP09LSCgoKHBwcmPZ+WDdEIpEYGRllZ2crGG9jYxMZGdm2PSQkxNLSMjAw8OTJk3V1dbm5uYsXLy4pKQkPD2euEFFESEiIlZWVn5/f6dOna2pqqqqqIiMjd+zYERYWxp4usXTpUoomssNMAAAgAElEQVSi7t27114nhw4d2r59+5UrV0Qikfw1JvKLqqqpqX399ddVVVVvv/12aWlpZWXlypUr8/LyoqKimItlFKRIP4qPxazq6urqqngCAACgAtqePQgAAANPXFwc5nzomJeXl5eXV1efxdw8krVkyZJWK2tu2rSJ/t+rP+bMmcM8VyKRmJub37p1y83NTSQSCQQCR0fHS5cuyfdfXV29fPlyU1NTgUAwffr0jIwMdq3TDRs2MDE5OTkODg5CodDCwuLAgQPscx0cHPT19S9fvty9A0IIiYuL6zRs48aNGhoaRUVFzMPy8nL5nbW1tW37lPfee8/AwKBVY0VFRWBgoKWlpaamplgsdnNzS0lJYTYpfkgrKyvXrFkzcuRITU3NoUOHurq6Jicny4/i5OSko6MjlUrb250OTpRIS0uTj/zzzz9nz56tq6uro6Pj5OTU6oWjaTopKaltJ1FRUa3COu1HwRjmBhzNzc3t7RoL8yEAgAqh6L5cAAwAAJREfHy8r68v5nzogLe3NyEkISGh30acMGFCRUVFN5a06B8URcXFxfn4+HQcVlNTY2Nj4+7ursiCoNyqrq42MzNbsmRJVFQU17n0suzs7IkTJx45cmThwoWdBmM+BABQIbg2BAAA2nX06FHmtPAune+tPH7++ecxY8awZ8V3O6YDOjo68ufPq6mp6evrSySS999/v9VdA2GgEovFSUlJiYmJBw4c4DqXjtA0HRAQoKuru3PnTq5z6WUFBQWenp7BwcGKFCwAAEC1oGYBAADtWrhwIU3TXVoIQEnk5+fPmzcvODiYWXmh2zGdqq+v/+uvvwghHh4eNE23tLTk5OTs2LEjJydn8uTJb7/99uPHj7vdOaiKiRMnZmZmnj59ura2lutc2lVWVlZQUJCSkqLgQiQqJDIycvfu3bt37+Y6EQAA6H2oWQAAwAC0ZcuWadOmXb16VSQS9SSmq9TV1Y2NjT08PM6ePfvRRx8dOnRo0aJFOAW9rbCwMIqisrOzi4qKKIravHkz1xn11IgRI06ePKmrq8t1Iu0yMTG5dOmSjY0N14n0vtDQUJxhAQAwUHXzVFgAAABl9vXXXwsEgp7H9MTevXsvXLhw4sSJo0ePLlq0qO8GUkXr1q1bt24d11kAAACAssN5FgAAMAApUozo04IFIYSiqFWrVhFCIiIi+nQgAAAAgIEKNQsAAPgfOTk58+fPF4vFQqHQwcHh0qVLbWPKy8sDAgJGjBjB4/GGDh3q6emZlZXFbDp+/Dh7Q8r79+/7+vrq6ekZGBi4u7vn5+ezPTQ1NX388cdjx47V1tYeMmTI3LlzT5w48fTpU0WGUCHTp08nhKSnp7e0tDAtOHQAAAAAikPNAgAA/uvu3bv29vaZmZmJiYllZWURERE7d+6U/8JMCCkpKbGzs4uPj4+IiKiqqjp//nxVVZW9vX1aWhohZP78+TRNe3h4EEICAwMDAwOLiori4uLOnj0rf33EqlWrvvjiiy+//LKysvL27dtjx4718PC4ePGiIkP0JycnJwMDg/T09O49nbnZoVQqraioIIPs0AEAAAD0AhoAAAaBuLg4ReZ8b29vQkhiYiLbUlRUxOfz+Xw+2/Lmm28SQmJjY9mWkpISPp9va2vLtjBfvJOSktgWLy8vQkh5eTnz0NLSctq0afJDjxkz5ty5c4oPoSBzc3N1dfVuxzg6Ourr61++fLmDp8uvG9IKu2hIcXExrfSHzsvLy8vLq9OwwYMQEhcXx3UW0PsUnA8BAEAZ4B6cAADwX2fOnCGEuLm5sS1mZmZjxozJzc1lW44fP66mpubu7s62mJiY2NjYXL16tbCwcNiwYWy7nZ0d+28LCwtCSHFxsaGhISFk1qxZ//73v999910/Pz87Ozt1dfU7d+50Y4i+dv78+Z48vaSkhBCiqanJ7LXyH7r09HSmbgWMzz77LCEhgessoJcVFhZynQIAACgK14YAAMD/aWpqqqur09LS0tHRkW83MjKSj6mpqZHJZGKxmJLz559/EkLy8vLknygWi9l/83g8QohMJmMeHjhw4PDhwwUFBc7Ozrq6urNmzTp27Fg3hlByzN1A7O3tNTU1cegAAAAAugrnWQAAwP/h8/kikaiurq6+vl6+bFFVVSUfo6enV19f/+TJEw2N7v8RoShq2bJly5Yta2lpOX/+fFhYmKen5yeffLJmzZreGoJzMpnswIEDhJCVK1cSFTl0U6dOxWkFLIqiPvzwQx8fH64TgV4WHx/v6+vLdRYAAKAQnGcBAAD/NXv2bPKfK0QYFRUV8pceEEI8PT2lUmlqaqp8Y2ho6PDhw6VSqYID6enp5eTkEEI0NTVdXFyYJTNOnTrVi0NwLjg4+I8//liwYAF7tQUOHQAAAECXoGYBAAD/tWfPniFDhgQGBiYnJ9fX19+6dWvp0qWtLhUJCQmxsrLy8/M7ffp0TU1NVVVVZGTkjh07wsLCuvTb/r/+9a9r1641NTU9evRo3759NE07OTn17hA919V1Q2Qy2aNHj3766SdnZ+d9+/b5+fnFxsZSFMVsHVSHDgAAAKAXcH0TUAAA6A+K3yf/zp078+fP19XVFQgEdnZ2J0+edHZ2Zv5kvPPOO0xMZWXlmjVrRo4cqampOXToUFdX1+TkZGZTqzU1N23aRNO0fMucOXNoms7KyvL393/++ee1tbWHDBkyderUqKgomUzGptHBEIpISkpq+ycvKiqqqzEODg4drxsiFArln05RlFgsHj9+/HvvvXf16tW28cp86LBuSCsE64YMUFg3BABAhVD0/34eAgCAAYm5fhtzPnSAuYYF97NgURQVFxeH+1kMPJgPAQBUCK4NAQAAAODGgwcP5s2bV1tbW1FRwa7zMnHixMbGRvkw+a0URU2ePJmrhDvW0tLy2Wef2draikQiIyOj2bNnJyUltVcamDdvHkVRu3btkm8MCgpiToIAAABgoGYBAAAAwIGsrKzJkye7urrq6uoaGhrSNJ2RkcG0BwYGykcyW9PS0gwMDGiazszM5CjljjQ0NDg5OR06dOizzz579OhRZmamjo7OvHnzbt682Tb48OHDz7w4a8WKFcHBwVu2bOn7fAEAQDWgZgEAACqGat+2bdu4zg4UpaOjM336dNXtv4dqa2vnzp37+uuvr1q1Sr6dz+cbGBhERkb+8MMPXOXWPevXr7927dqvv/76yiuvCASC4cOHHzp0iM/nt40sLi4ODAxctmxZ201WVlbHjh3bvXt3fHx836cMAAAqADULAABQMR3cpQk1C1AV+/btKy0t/fjjj1u1a2lpxcbGqqmp+fv75+bmcpJbN5SVlR08eHDJkiXGxsZso1AobGxsHDduXKvgFStWeHt7u7q6PrMriUTi5eW1du1arM4LAAAENQsAAACAfkbTdHR09JQpU8zMzNpudXNz27x5c11dnbe3d6sbWyitEydOPH36VJETW2JiYm7evBkWFtZBzIIFCwoLC0+dOtV7CQIAgKpCzQIAAAC6hllO1crKisfj6evrz549+9y5c8ymXbt2MdfpsF9fz5w5w7QYGhoyLWFhYRRFNTQ0pKamMps0NDTYdoqihg0blpGR4ezsLBKJtLW1Z8yYkZqa2vP+lUd2dnZZWZlEImkvYOvWra6urteuXfvggw867qqD1+L48ePsZVP379/39fXV09MzMDBwd3fPz8+X76S8vDwgIGDEiBE8Hm/o0KGenp5ZWVld2qM///yTEKKvr7927VoLCwsej/fcc88FBARUVVXJhxUWFq5duzYmJkYkEnXQ24QJEwghv/zyS5dyAACAAQk1CwAAAOiC0tJSOzu7I0eOhIeHV1RUXLlyRVtb29nZOTo6mhCyefNmmqaFQiEbP2vWLJqmbW1t2ZZ169YxMS+//DJzUQ9zFQDTLpFIqqurV69evWvXrtLS0t9//72qqsrJyenChQs97J/h5ORkYGCQnp7elwepEzdu3CCEDBs2rL0ANTW12NhYCwuL6Ojo2NjY9sI6fi3mz59P07SHhwchJDAwMDAwsKioKC4u7uzZs4sWLWI7KSkpsbOzi4+Pj4iIqKqqOn/+fFVVlb29fVpamuJ7VFJSQgjx8/MrKyu7cOHCo0ePdu7cGRMTY29vX1NTw4YtX7588eLFTk5OHfdmbm5O/nOUAABgkEPNAgAAALogODj43r17n3/+ubu7u66u7pgxY44cOWJqahoQEFBWVtYrQzQ0NERERNjb2wuFwsmTJ3///ffNzc2rV6/ulc5lMhlTyOiV3rqH+YYvFos7iDE0NIyPj9fU1PT398/JyXlmjOKvxfLly5njOXPmzDlz5mRkZFRUVLCdPHjw4NNPP33ttdd0dHRsbGyOHj1K03Snp3jIY65hEQgEhw4dGjlypJ6e3htvvBEcHJybm/vJJ58wMVFRUXl5efv27eu0N11dXYqimKMEAACDHGoWAAAA0AXHjh0jhMyZM4dt4fP5zs7OT5486a2T+YVCIXN1AGP8+PFmZmbZ2dm98iWWPY+g5111G/MNX1NTs+OwqVOnhoWFNTQ0eHt7P3nypG2A4q+FnZ0d+28LCwtCSHFxMfPw+PHjampq7u7ubICJiYmNjc3Vq1cLCwsV3CPmzJeZM2fKX4Yzd+5c8p9LPP7+++/169fHxMTInyPTAQ0NjWfuMgAADDaoWQAAAICimpqaampqtLS0Wt2PgFktorS0tFdG0dPTa9ViZGRECHn06FGv9M85LS0tQkhLS0unkQEBAb6+vjdu3Gi1JCrp4mshf04Hj8cjhMhkMrYTmUwmFovllw1m7k+Rl5en4B6NGDGCEGJgYCDfyLxq5eXlhJCkpKSamppXX32VHYJZ63TLli3Mw7t378o/VyqVCgQCBUcHAIABDDULAAAAUBSfzxeLxY2NjXV1dfLtzJUIJiYmzEM1NbXm5mb5gOrq6lZdURTV3iiVlZWtrt1gqhXMd+Ce9885U1NTQoj8jR46EB0dbW1tHRMT891338m3K/hadIzP5+vp6WloaLS0tLRdPHjGjBkK7hFzS9RWJ8IwrxpTQ1m5cmWrzpnd2blzJ/Nw1KhR7BNra2tpmmaOEgAADHKoWQAAAEAXLFiwgBAivw5lU1NTSkqKQCBwc3NjWkxNTYuKitiA0tLSv//+u1U/2trabN3B2tr64MGD7KbGxsaMjAz24fXr14uLiyUSCfsltof9c27cuHGEEAWvvNDR0fnxxx+FQmFERESrTYq8Fp3y9PSUSqXsyiyM0NDQ4cOHy9+7tGOvvfaaubn5mTNn5BdnTUpKIoTMnz9fwU5YzIvLHCUAABjkULMAAACALggJCbG0tAwMDDx58mRdXV1ubu7ixYtLSkrCw8OZX9QJIa6ursXFxfv376+vr8/Pz1+9ejV7igRr0qRJubm5Dx8+TEtLKygocHBwYDeJxeKNGzempaU1NDRkZmYuXbqUx+OFh4ezAT3pXxnWDZFIJEZGRtnZ2QrG29jYREZGtm1X5LXoVEhIiJWVlZ+f3+nTp2tqaqqqqiIjI3fs2BEWFsbenGLp0qUURd27d6+9Tvh8fnR0dGVl5cKFC/Py8qqrq7/77ruQkJApU6YEBAQomAmLWWnV1dW1q08EAIABqO15gAAAMPDExcVhzoeOeXl5eXl5KRJZUVERGBhoaWmpqakpFovd3NxSUlLkA6qrq5cvX25qaioQCKZPn56RkcGuRbphwwYmJicnx8HBQSgUWlhYHDhwgH2uRCIxNze/deuWm5ubSCQSCASOjo6XLl3qrf4dHBz09fUvX77c6W4SQuLi4hQ5IN2wceNGDQ2NoqIi5iFz0weWra1t26e89957BgYGrRo7eC1aLVa6adMm+n+vuJkzZw4TWVlZuWbNmpEjR2pqag4dOtTV1TU5OVl+FCcnJx0dHalU2vFOXb582c3NTSwW83i8sWPHbtu27fHjx23D/P39W30cdXNzkw/w9vY2Nzdvbm7ueLhuw3wIAKBCKJrTtb4AAKB/xMfH+/r6Ys6HDnh7exNCEhISuE1jwoQJFRUViq9Y0XcoioqLi/Px8emLzmtqamxsbNzd3b/66qu+6L8XVVdXm5mZLVmyJCoqqh+Gy87Onjhx4pEjRxYuXNhHQ2A+BABQIbg2BAAAAKC/icXipKSkxMTEAwcOcJ1LR2iaDggI0NXV3blzZz8MV1BQ4OnpGRwc3HcFCwAAUC2oWQAAAABwYOLEiZmZmadPn66treU6l3aVlZUVFBSkpKQouBBJD0VGRu7evXv37t39MBYAAKgE1CwAAABAKYSFhVEUlZ2dXVRURFHU5s2buc6oz40YMeLkyZO6urpcJ9IuExOTS5cu2djY9M9woaGhOMMCAADkaXCdAAAAAAAhhKxbt27dunVcZwEAAABKBOdZAAAAAAAAAIAyQs0CAAAAAAAAAJQRahYAAAAAAAAAoIxQswAAAAAAAAAAZYSaBQAAAAAAAAAoI4qmaa5zAACAPhcfH+/r68t1FgAAygKfgQEAVALWOgUAGBSmTZsWFxfHdRbQa65fv3706NH8/HxnZ+cVK1Zwnc5A9u23354+fXr48OELFy6cNGkS1+kAAAAMLjjPAgAAQJX88ccfGzduTElJmTlz5t69e21tbbnOaOC7ffv21q1bExMTp0yZsmvXLmdnZ64zAgAAGCxwPwsAAADVcOvWLR8fn6lTpz5+/PjcuXPJyckoWPSP559/Pj4+Pisry8LCYubMmdOnT7948SLXSQEAAAwKqFkAAAAouwcPHvj7+7/44ou3b9+Oi4u7fPnyq6++ynVSg86LL74YHx+fmpqqqan5yiuvuLi4/PXXX1wnBQAAMMChZgEAAKC8ysvLg4KCrK2tL1y4EBMTk52d7e3tzXVSg9q0adOYk1z++eefyZMn+/j45OXlcZ0UAADAgIWaBQAAgDKqqqoKCgp67rnnYmNjv/jiixs3brzxxhtqavjDrRRmzpyZkZFx/PjxO3fu2NjYvPHGG/fu3eM6KQAAgAEI9+AEAABQLg0NDfv379+7d6+6uvr69etXr16tpaXFdVLwbDKZ7McffwwODn748OFbb721fft2ExMTrpMCAAAYOFCzAAAAUBbNzc2HDh3aunVrfX39ypUrg4ODxWIx10lB51paWr755pvt27fX1NSsWrUqKChIT0+P66QAAAAGAtQsAAAAuMf8XB8UFFRYWPjWW2/t3LnTyMiI66Sgax4/fhwVFbVnzx6pVPrBBx+sXbtWJBJxnRQAAIBqQ80CAACASzRNJyYmbtmypaCg4O233966dauZmRnXSUH31dfXHzhwICQkhMfjrV27Fpf2AAAA9ARu5QUAAMCZ3377zc7ObuHChS+++OLNmzcjIyNRsFB1Ojo6GzZsyM/P9/Pz2759+5gxYw4ePCiVSrnOCwAAQCWhZgEAAMCBtLQ0JycnFxcXfX39q1evxsfHjx49muukoNcYGBjs3bs3Nzd3wYIFAQEB48aNO3z4sEwm4zovAAAAFYOaBQAAQL+6ceOGj4/PtGnTmpubL1y4kJycPGHCBK6Tgj4xbNiw8PDwO3fuODo6+vn5SSSShIQErpMCAABQJahZAAAA9JP79+/7+/tLJJL79++fOHHi0qVLr7zyCtdJQZ977rnnIiMjr1279vzzz/v6+trb2589e5brpAAAAFQDahYAAAB9rrCw0N/ff/To0RcvXjx69OiVK1fmzp3LdVLQr1544YX4+Pj09HRDQ0NnZ2cXF5eMjAyukwIAAFB2qFkAAAD0ocrKyqCgoDFjxpw5c+bAgQPXr1/39vamKIrrvIAbL730UlJS0qVLl1paWl566SUXF5esrCyukwIAAFBeqFkAAAD0ifr6+tDQUCsrq5iYmK1bt+bm5r777rvq6upc5wXce/nll8+fP5+cnFxVVWVra+vj45OXl8d1UgAAAMoINQsAAIBe1tzcfPDgQSsrq9DQ0MDAwPz8/A0bNvD5fK7zAuUyc+bMzMzMo0ePXrt2zcbGxt/fv6ioiOukAAAAlAtF0zTXOQAAAAwQLS0t33zzzY4dO6qrq1etWrVhwwZ9fX2ukwJlJ5PJfvzxx6CgoMLCwrfeemvHjh3GxsZcJwUAAKAUcJ4FAABAL5DJZAkJCTY2NqtWrZozZ05eXt7evXtRsABFqKmpeXt73759+8svv0xKSrKysgoKCqquruY6LwAAAO7hPAsAAICe+u233z766KPs7OzXX389JCTEysqK64xAVT1+/PjLL78MDQ2lKOqDDz5Yu3atSCTiOikAAADO4DwLAACA7ktNTXV0dHRxcTEwMPjrr7/i4+NRsICe0NbW3rBhw4MHDz766KPPPvuMuStKY2Mj13kBAABwAzULAACA7sjIyJg7d+706dN5PF5GRkZycvKLL77IdVIwQIhEog0bNuTn5/v5+W3fvt3a2vrgwYNSqZTrvAAAAPobahYAAABdc/v2bR8fnylTplRUVKSkpCQnJ0+ePJnrpGAAMjQ03Lt3b25u7qxZs1auXDl+/PjDhw/LZDKu8wIAAOg/qFkAAAAo6u+///b39x8/fvytW7fi4uIuX77s5OTEdVIwwA0bNiwyMjIvL++VV17x8/OTSCQJCQlcJwUAANBPULMAAADoXHl5eVBQkLW19a+//hoREZGdne3t7U1RFNd5wWAxYsSIyMjIa9euPf/8876+vtOmTTt37hzXSQEAAPQ51CwAAAA68s8//2zbts3Kyio2Nnbv3r05OTnvvvuuuro613nBYPTCCy/Ex8enp6cbGBg4OTm5uLhkZmZynRQAAEAfQs0CAADg2RoaGkJDQ62srPbv379p06bc3NzVq1fz+Xyu84LB7qWXXkpKSrp48WJzc7OdnZ2Li0t2djbXSQEAAPQJ1CwAAABaa2lpOXjw4OjRo3ft2vXuu+/m5+dv2LBBIBBwnRfAf02fPv3ChQvJyclVVVWTJk3y8fG5e/cu10kBAAD0MtQsAAAA/ksmkyUkJDz//PMffPDB3Llz7969u3fvXrFYzHVeAM82c+bMzMzMo0ePZmdnv/DCC/7+/kVFRVwnBQAA0GtQswAAAPg/v/3228SJExcvXjxt2rScnJzIyEhjY2OukwLoBEVR3t7et2/fjo2N/e2330aOHOnv719WVsZ1XgAAAL0ANQsAAADy22+/2dnZubq6Wltb37x58/Dhw5aWllwnBdAFampqTOXiyy+/TEpKGjVqVFBQUHV1Ndd5AQAA9AhqFgAAMKilp6c7Ozu7uLjo6ellZmbGx8ePGTOG66QAuonH47377rt5eXmbN28+ePCglZVVaGjokydPuM4LAACgm1CzAACAQermzZs+Pj7Tpk1rbGw8f/58cnLypEmTuE4KoBcIhcINGzY8ePDgo48+2rNnz3PPPRcaGtrU1MR1XgAAAF2GmgUAAAw6Dx488Pf3l0gkOTk5cXFxqampjo6OXCcF0MtEItGGDRvy8/P9/Py2bds2ZsyYgwcPPn36lOu8AAAAugA1CwAAGESKiopWr15tbW39+++///DDD9nZ2d7e3lwnBdCHDA0N9+7dm5ubO2vWrJUrV44fPz4hIYGmaa7zAgAAUAhqFgAAMChUVlYGBQWNHj362LFjX3zxxfXr1729vSmK4jovgP5gYWERGRmZl5fn4OCwaNEiiUSSkJDAdVIAAACdQ80CAABUXnFx8ZUrV9rbWl9fHxoaamVl9fXXX2/dujU3N/fdd9/V0NDozwwBlMGIESMiIyOzs7PHjh3r6+s7bdq0c+fOdRD/888/91tuAAAAz4SaBQAAqLaqqionJ6f33nuv7enuzc3NBw8eHDVq1O7du//1r3/l5+dv2LBBS0uLkzwBlISNjU18fHxaWpq2traTk5OLi0tmZmbbsOzs7Llz5+7Zs6f/MwQAAGChZgEAACqsvr7e1dU1Pz8/Kyvr2LFjbLtUKj18+PDYsWM//PBDX1/fu3fv7t27V1dXl8NUAZTKlClTfvvtt4sXLzY3N7/00ktz5869du2afMDGjRsJIZs2bYqIiOAoRwAAANQsAABAZTU3N8+fPz87O1sqlVIUtX79eqlUStN0QkKCjY3N8uXLXVxc8vLywsPDjYyMuE4WQBlNnz79woULv/76a1FR0cSJE318fO7evUsI+eOPP06fPi2TyQghq1atiomJ4TpTAAAYpCjcOBoAAFTR06dPfXx8Tpw4IZVKmRY1NbV169b99ttvWVlZr7/++p49e0aNGsVtkgCqQiaT/fjjj5s3b753797bb7997dq1zMxM+TdXXFycl5cXt0kCAMAghJoFAACoHpqmly9f/u233z59+pRtpCiKx+O9/PLLn376qUQi4TA9ABUllUqPHDmyYcOG0tJS+XaKotTV1U+ePOnm5sZVbgAAMDjh2hAAAFA969evP3TokHzBghBC07RUKvX09ETBAqB7NDQ03njjDTMzs1YL69A0LZPJPDw8UlNTucoNAAAGJ5xnAQAAKmb79u3bt29v7++Xvr7+gwcPRCJRP2cFMDAcP358wYIFz9ykrq4uEAguXrw4YcKEfs4KAAAGLZxnAQAAqiQiImLbtm0dFNzr6ur279/fnykBDBgymWzTpk3q6urP3Pr06dPGxkZnZ+c7d+70c2IAADBo4TwLAABQGd99992bb77ZwV8uiqIoihKJRPfu3dPX1+/P3AAGgCNHjixZskRNTY1ZMeSZNDQ0jI2Nr1y5Ym5u3p+5AQDA4ISaBQAAqIYTJ054enrK38OCueSeWdpAQ0PD3Nx83Lhxzz///OjRo93d3c3MzDjLFUA1nT179vLly7m5ubdv387Ly6upqSGEqKmp8Xi8lpYW9t2npqY2cuTItLQ0Q0NDTvMFAICBDzULAJWXlpb26aefcp0FQN8qLy+/ePEi+9uvlpaWSCTS1dXV0dERiUQ6OjpCoZCiKG6THCTs7e3XrFnTw04+/fTTtGjcsPMAACAASURBVLS0XskH+k5zc3N9fX19fX1dXV19fX1tbW1DQwO7AKqenp6jo6Ompia3SQIoszVr1tjb23OdBYBq0+g8BACU28OHDxMTE728vLhOBPpJYmLi1KlThw0bxnUi/ae5ubm4uPiFF15gyhM6Ojrs9faFhYWXLl3C//9+k56e3iv9pKWlpaenT506tVd6gz7C4/GGDBkyZMgQ+campiamhFFfX//gwYNRo0ZxlV6nCgsL09PTMT+wmPcv3nf9JjEx0dvbGzULgB5CzQJggEhISOA6BegnFEV9+OGHPj4+XCeiFOLj4319ffH/v994e3v3VldTp07FCwd9CvNDK8z7Fwek3+DsP4BegXVDAAAAAAAAAEAZoWYBAAAAAAAAAMoINQsAAAAAAAAAUEaoWQAAAAAAAACAMkLNAgAABpEHDx7Mmzevtra2oqKC+o+JEyc2NjbKh8lvpShq8uTJXCXcsZaWls8++8zW1lYkEhkZGc2ePTspKam9VcznzZtHUdSuXbvkG4OCguLi4volWQBQMZgwMWECKAPULAAABoX6+vrRo0e7u7tznQiXsrKyJk+e7Orqqqura2hoSNN0RkYG0x4YGCgfyWxNS0szMDCgaTozM5OjlDvS0NDg5OR06NChzz777NGjR5mZmTo6OvPmzbt582bb4MOHDyclJbVtX7FiRXBw8JYtW/o+XwCVgQmTYMLEhAmgNFCzAAAYFGialslkMpmMqwR0dHSmT5/O1eiEkNra2rlz577++uurVq2Sb+fz+QYGBpGRkT/88ANXuXXP+vXrr1279uuvv77yyisCgWD48OGHDh3i8/ltI4uLiwMDA5ctW9Z2k5WV1bFjx3bv3h0fH9/3KQOoBkyYmDAxYQIoD9QsAAAGBZFIlJ+f//PPP3OdCGf27dtXWlr68ccft2rX0tKKjY1VU1Pz9/fPzc3lJLduKCsrO3jw4JIlS4yNjdlGoVDY2Ng4bty4VsErVqzw9vZ2dXV9ZlcSicTLy2vt2rVSqbQPMwZQHZgwMWFiwgRQHqhZAADAwEfTdHR09JQpU8zMzNpudXNz27x5c11dnbe3d6vrtJXWiRMnnj59qsgvsTExMTdv3gwLC+sgZsGCBYWFhadOneq9BAFAVWHCxIQJoFRQswAAGPiOHz/O3h2N+Ygp33L//n1fX189PT0DAwN3d/f8/HzmWWFhYUzAsGHDMjIynJ2dRSKRtrb2jBkzUlNTmZhdu3YxMexnwTNnzjAthoaG8v00NDSkpqYymzQ0NPr5CGRnZ5eVlUkkkvYCtm7d6urqeu3atQ8++KDjriorK9esWWNlZcXj8fT19WfPnn3u3DlmkyJHlVFeXh4QEDBixAgejzd06FBPT8+srKwu7dGff/5JCNHX11+7dq2FhQWPx3vuuecCAgKqqqrkwwoLC9euXRsTEyMSiTrobcKECYSQX375pUs5AAxImDAxYWLCBFAuNACoOOYW1lxnAf2HEBIXF9eNJ3p4eBBCnjx50qrFw8Pj8uXL9fX1ycnJAoHAzs5O/lkSiUQoFNrb2zMxGRkZL774Io/HO3/+PBsjFApffvll+WfZ2toyN2PrIIYxY8aMIUOGpKWldWOPaIX//3/33XeEkD179rRqz8jIEIvFzL/Ly8stLCwIId9//z3Twt5SjlVSUmJpaWlsbJyUlFRTU3Pnzh1PT0+KoqKiotiYTo9qcXHxc889Z2xsfOrUqbq6uhs3bjg6OmppaV2+fFnxHWdGMTExWbJkSX5+/j///PPtt98KhcIxY8ZUV1ezYW5ubu+//778Qdi5c2fb3mpqagghDg4OnY7r5eXl5eWleJ593Q9AB3ry93FATpgKvu8wYdK9NGF2++81AMjDeRYAAIPd8uXL7e3thULhzJkz58yZk5GRUVFRIR/Q0NAQERHBxEyePPn7779vbm5evXp1r4wuk8mYP0i90lt7SkpKCCFisbiDGENDw/j4eE1NTX9//5ycnGfGBAcH37t37/PPP3d3d9fV1R0zZsyRI0dMTU0DAgLKysrkIzs4qsHBwQ8ePPj0009fe+01HR0dGxubo0eP0jTd6S+W8piffwUCwaFDh0aOHKmnp/fGG28EBwfn5uZ+8sknTExUVFReXt6+ffs67U1XV5eiKOYoAUAHMGEyMGFiwgToN6hZAAAMdnZ2duy/md/NiouL5QOEQiFzKixj/PjxZmZm2dnZvfKJ7fz581VVVfb29j3vqgPMB1ZNTc2Ow6ZOnRoWFtbQ0ODt7f3kyZO2AceOHSOEzJkzh23h8/nOzs5PnjxpdZ5wB0f1+PHjampq8ssompiY2NjYXL16tbCwUME9EgqFhJCZM2fKnzc+d+5c8p8zlv/+++/169fHxMQwkZ3S0NB45i4DgDxMmCxMmAqODgA9hJoFAMBgJ/9jGo/HI4S0WuFPT0+v1VOMjIwIIY8ePer77HqHlpYWIaSlpaXTyICAAF9f3xs3brRa4Y8Q0tTUVFNTo6Wl1epSZ+ZG9KWlpfKN7R1VphOZTCYWiyk5zOXWeXl5Cu7RiBEjCCEGBgbyjczrUl5eTghhTsZ+9dVX2SGYpfu2bNnCPLx79678c6VSqUAgUHB0gEELE6Y8TJgA0A9QswAAgE5UVla2OhWZ+fDNfOAjhKipqTU3N8sHVFdXt+qEoqi+zLETpqamhBDmIuRORUdHW1tbx8TEMNczs/h8vlgsbmxsrKurk29nTnI2MTFRpHM+n6+np6ehodHS0tL2is0ZM2YouEfMPfxa/XLLvC7MV4KVK1e26rzV5dmjRo1in1hbW0vTNHOUAKAnMGEyMGECQG9BzQIAADrR2NiYkZHBPrx+/XpxcbFEImE/sZmamhYVFbEBpaWlf//9d6tOtLW12Y/p1tbWBw8e7OOs/8e4ceMIIQqeSKyjo/Pjjz8KhcKIiIhWmxYsWEAIkV/irqmpKSUlRSAQuLm5KZiMp6enVCpllxJghIaGDh8+XCqVKtjJa6+9Zm5ufubMGfm1BpOSkggh8+fPV7ATFvPyMUcJAHoCEyYLEyYA9ArULAAAoBNisXjjxo1paWkNDQ2ZmZlLly7l8Xjh4eFsgKura3Fx8f79++vr6/Pz81evXs3+osiaNGlSbm7uw4cP09LSCgoKHBwcmHYnJycDA4P09PQ+3QWJRGJkZJSdna1gvI2NTWRkZNv2kJAQS0vLwMDAkydP1tXV5ebmLl68uKSkJDw8nPmxThEhISFWVlZ+fn6nT5+uqampqqqKjIzcsWNHWFgYe6310qVLKYq6d+9ee53w+fzo6OjKysqFCxfm5eVVV1d/9913ISEhU6ZMCQgIUDATFrNwoKura1efCACtYMJkYcIEgN7R9aVGAEC5YK3TwYZ0fe005kZorCVLlqSlpcm3bNq0if7fk5nnzJnDPFcikZibm9+6dcvNzU0kEgkEAkdHx0uXLsn3X11dvXz5clNTU4FAMH369IyMDFtbW6afDRs2MDE5OTkODg5CodDCwuLAgQPscx0cHPT19bu0ap08xf//b9y4UUNDo6ioiHnIXMPMsrW1bfuU9957r9XSfTRNV1RUBAYGWlpaampqisViNze3lJQUZpPiR7WysnLNmjUjR47U1NQcOnSoq6trcnKy/ChOTk46OjpSqbTjnbp8+bKbm5tYLObxeGPHjt22bdvjx4/bhvn7+7f66+/m5iYf4O3tbW5u3tzc3PFwNNY6BZXSvb+PA3jCVPx9hwmzVyZMgrVOAXoDRffxakkA0Nfi4+N9fX3xXh48KIqKi4vz8fHpn+EmTJhQUVGh+O3Z+5ni//9rampsbGzc3d2/+uqrfkisJ6qrq83MzJYsWRIVFdUPw2VnZ0+cOPHIkSMLFy7sNNjb25sQkpCQ0MNBe6sfgA70/99HJZ8wFX/fYcJsT5cmzH7+ew0wUOHaEAAAGBTEYnFSUlJiYuKBAwe4zqUjNE0HBATo6uru3LmzH4YrKCjw9PQMDg5W5PM3AAwSmDCfCRMmACdQswCAgU9HR0d+jTQ1NTV9fX2JRPL+++9fvXqV6+yg/0ycODEzM/P06dO1tbVc59KusrKygoKClJQUBe+r30ORkZG7d+/evXt3P4zVc0ePHmXexcxajNAXMGECAxNmW6o1YQIMGKhZAIBC6uvrR48e7e7uroS9KTLcX3/9RQjx8PCgabqlpSUnJ2fHjh05OTmTJ09+++23Hz9+3D+ZqJawsDCKorKzs4uKiiiK2rx5M9cZ9YIRI0acPHlSV1eX60TaZWJicunSJRsbm/4ZLjQ0VIV+MFy4cCFN087OzlwnMpBhwuweTJicwIQJMBigZgEACqFpWiaTyWSyvutNR0eHWUG9r6mrqxsbG3t4eJw9e/ajjz46dOjQokWLcEOQttatWyd/A6Rdu3ZxnRGAkuq36UtxvZUSJkwFYcIEAOgjqFkAgEJEIlF+fv7PP/+shL31xN69e6dMmXLixImjR49ynQsAgFLDhAkAAP0PNQsAGNQoilq1ahUhJCIigutcAACUGiZMAADof6hZAAwKzHW2FEUNGzYsIyPD2dlZJBJpa2vPmDEjNTWViTl+/Dh707U7d+74+PgYGBgwD6Ojo9lNjY2NbLfMkulWVlY8Hk9fX3/27Nnnzp3rRm9Meg0NDampqUy7hoZGdXW1/H3gmPNspVIp2+Ll5dUrB4c5fTo9Pb2lpYVpKS8vDwgIGDFiBI/HGzp0qKenZ1ZWVtv9un//vq+vr56enoGBgbu7e35+PttnU1PTxx9/PHbsWG1t7SFDhsydO/fEiRNPnz5lAzoYAgDk5eTkzJ8/XywWC4VCBweHS5cutY3p4A2lyOxHCJFKpXFxcS4uLiYmJgKBYPz48eHh4ez1ax1MaEFBQW2nr1ZPefDgga+vr0gkMjAwWLZs2T///HP//v25c+eKRCJTU9MVK1bU1dUpuDuKTEHPnFF76+XAhAkAAP2NBgAVFxcXp+B7WSKRCIVCe3v7y5cv19fXZ2RkvPjiizwe7/z582yMh4cHIcTR0fHcuXMNDQ3p6enq6url5eXspidPnjCRJSUllpaWxsbGSUlJNTU1d+7c8fT0pCgqKiqqG73RNC0UCl9++eVWObu5uampqd29e1e+0d7ePjY2ln04Y8aMIUOGpKWldbDv8reUa+XJkyfMfFhcXEzTdHFx8XPPPWdsbHzq1Km6urobN244OjpqaWldvny51X55eHgwRzI5OVkgENjZ2bEBy5cvF4vFv/766+PHj0tLS9etW0cIOXfuHLNVkSE6QAiJi4tTJHIwUPz/P/QKLy8vLy+vfusnLy9PT0/P3Nz8119/rauru3btmqur64gRI/h8PhujyBuq09kvKSmJELJnz56qqqry8vIvvvhCTU2t1R0KOpjQnjl9sU/x9PTMzMysr68/fPgwIWT27NkeHh5//fVXXV3dV199RQj58MMPu7Q7nU5BHaQ0qCZMzA+t9Nb7FxSEv9cAvQLzOIDK61LNghDy119/sS3Xrl0jhEgkEraF+XD5888/t316qyrDW2+9RQj54Ycf2IDGxkYzMzOBQFBaWtrV3uh2PmH/8ssvhJD333+fbbl06ZK5uXlzczPb4ujoqK+v3/Hn1w4+grP3wGc+gr/55puEEPmaSElJCZ/Pt7W1bZV8UlIS28Kc9MF8e6Fp2tLSctq0afKjjBkzhv0IrsgQHcBnIHn4TtLP+rlm4e3tTQhJTExkW4qKivh8vnzNQpE3VKezX1JS0quvvio/9NKlSzU1NWtqatiWDia0jmsWp06dYluYBQ4uXLjAtlhaWlpbW3dpdzqdgjpIaVBNmJgfWkHNop/h7zVAr8A8DqDyunqeRatGMzMz9tMn/Z8PlxUVFW2f3qrKIBaLCSG1tbXyMcuWLSOEfPvtt13tjW7/E/b48eO1tbXZTjw8PPbu3avI/srr4CM4c4qypqYmUwcRi8VqamryX1Romp40aRIh5OHDh/LJs6UZmqY//PBDQkh2djbz8L333iOErFixIi0tTSqVthpRkSE60JVz6QB6X3/WLEQiESGkrq5OvnH8+PHyNQtF3lCKzH6t/L//9/8IIW3PF3jmhNZxzaKsrIxtcXFxIYQ0NDSwLdOnTxeJRF3anU6noA5SUsSAmTCZv48AHELNAqDneu36RgBQCXp6eq1ajIyMiouLHz16ZGpqyjYKhcKO+2lqaqqpqdHS0mK+UbCMjY0JIaWlpfKNnfbWscDAwHfeeSciImLLli25ublnz5795ptvetJhK8zl8fb29pqamsx+EUKYikwreXl5w4YNYx/Kx/B4PEIIe/X7gQMH7O3tv/32W2dnZ0KIg4ODv7//ggULyH8OnYJDtCcwMNDe3r4LOzlwpaWlff755/hm0m8+++yzfhurqamprq5OS0tLR0dHvt3IyCg3N5eNUfAN1fHsV1NT88knnxw7dqywsLC6upqNYU8rYHVjQtPV1WX/raampq6urq2tzbaoq6uzU0dvTUF9RxUnTMwPLOb9y9SMoB/4+vpynQLAQICaBcDgUllZSdM0RVFsy6NHjwghRkZGXeqHz+eLxeKampq6ujr5skVZWRkhxMTEpBu5yWclb8mSJRs3bty/f/9HH330ySefvPnmm/r6+t3o/5lkMtmBAwcIIStXriSE8Pl8PT29+vr6J0+e9OSudRRFLVu2bNmyZS0tLefPnw8LC/P09Pzkk0/WrFnTK0PY29v7+Ph0O70B5vPPP8fR6DcJCQn9NhafzxeJRHV1dfX19fJli6qqKvkYBd9QHc9+c+fOvXjxYnh4+KJFiwwNDSmK+vzzz5nbTCiSanvTV1f11hTUiynJU9EJE/MDi3n/4oD0G9QsAHoF1g0BGFwaGxszMjLYh9evXy8uLpZIJPInWSiI+RHs1KlTbEtTU1NKSopAIHBzc+tGbtra2s3Nzcy/ra2tDx48yPybz+e///77jx49+uSTT2JjY1evXt2NztsTHBz8xx9/LFiwgLlsnhDi6ekplUrlFxQghISGhg4fPlwqlSrYrZ6eXk5ODiFEU1PTxcWFuXk+e6x6ZQiAwWD27NmEkDNnzrAtFRUVd+7ckY9R8A3Vwez39OnT1NRUExOTgICAoUOHMt/22ZtNKqK96asbemt+6MWUWJgwAQCg/6FmATC4iMXijRs3pqWlNTQ0ZGZmLl26lMfjhYeHd6OrkJAQS0vLwMDAkydP1tXV5ebmLl68uKSkJDw8nLlCpKsmTZqUm5v78OHDtLS0goICBwcHdtP7778vEAg2b948c+bMUaNGtXqik5OTgYFBenq6ggPJZLJHjx799NNPzs7O+/bt8/Pzi42NZX+TDAkJsbKy8vPzO336dE1NTVVVVWRk5I4dO8LCwrr0K9+//vWva9euNTU1PXr0aN++fTRNOzk59e4QAAPenj17hgwZEvj/27v7mDaLOA7g9zAKlEILA8bb0DHMFoOzMFgcuoaxmTYKMkE6jMPEKITMF9YwjLCBS9yQbBLjEpkgiInOOboZSGBOQ8iWyFaTgqE6CWPC3CyFSZm8BgaExz8uNrVAeUYLT2m/n7/oPddf7zna39Nc77lTqZqbm8fHxzs7O7OysixuFeH4gbKS/datW7d79+6BgYGPPvrIaDROTk5euXKF7ujBkZX09bDslR8WaxISJgAArDG8rqYBAHbwUGtwhoeHd3Z2KhQKX19foVCYmJjY2tpKj2o0msXyQ319vXn5gQMHaLnRaFSpVJGRkQKBQCKRKBSKlpaWZUfr6uqSyWQikSgiIqKiosKi8Tk5OeT/i+2byGQy68vgW9x/zjCMRCLZtm3bwYMH29vb59cfGhrKz8/fvHmzQCAICgqSy+XNzc0LntfRo0fZ/08dT05OZlm2o6MjNzf38ccf9/b2Xr9+/c6dO6urq+fm5ri8xJII1vQyg30BVtkq7xvCsuzNmzdffPFFsVhMN8hsamqiqx4QQt544w1aZ8kPlPXsx7Ls4OBgbm5uRESEQCAIDg5+7bXXCgsL6avExcVZSWjU/PQ1P1eYz/IghJSVlf3000/mJceOHVvydDimoAWbRLlUwkR+sIB9Q1YZrtcAdsGwWIIeYI1Tq9WZmZlcPssxMTFGo1Gv169Cq+zuyy+/rKioaGtr47sh/GMYpq6uDjckU9zf/2AX9KYA21e1sFccjtZ09oNlQ36wsMqfO8D1GsAucG8IAKwNlZWV+fn5fLcCnNCdO3dSU1NHR0eNRiPzn9jY2KmpKfNq5kcZhomPj+erwUv6/vvvt2zZsuC0+cLCQuyhAADL5nwJk0pNTWUY5sSJE+aFSJgADgJjFgDguGpqatLS0sbHxysrK//55x/8UgF219HRER8fL5fLxWJxYGAgy7J0An9HR4dKpTKvSY9qNJqAgACWZR1zyk9PT09qampRURHdwWe+nJycoqKikpKSVW4YADgBJ0uYJl999VVjY+P8ciRMAAeBMQsAl1BeXs4wjE6n6+vrYximuLiY7xZx1dDQ4O/v/9lnn50/fx7Lra0+Hx+fXbt2rd341o2Ojr7wwgsvvfTS22+/bV7u6ekZEBBQVVX17bff8tW25SkpKXn66afb29vNdyA2FxUVVV9fX1paqlarV7ltfFm72Q/WHCRMvtpmC4PBoFKpXn311fmHXDBhAjgmjFkAuISCggLzlWwsZj86rOzsbJZlZ2ZmdDrd9u3b+W4OOJtTp04NDAy8//77FuVeXl7ffPONm5tbbm5ud3c3L21bni+++KKwsND66J5UKs3IyDh8+LCL7BO5RrMfgKNxvoRJ5eTkKJVKuVy+4FFXS5gAjgljFgAA4IpYlq2pqXnqqafCwsLmH1UoFMXFxWNjY0ql0uI+bUcmFAq5VEtLS9Pr9ZcuXVrp9gCAc3DKhEkIqa2t/f3338vLy63UQcIE4B3GLAAAnBPdHTAqKsrDw8Pf3/+55567cuUKPXTixAm6LpppmvEPP/xASwIDA2kJnVE/MTFx7do1eoj+ek/LGYbZuHGjVqvdu3evr6+vt7d3UlLStWvXbI+/anQ63b1796RS6WIVjh07JpfLf/3113feecd6KCtd3dDQYFqF7s8//8zMzPTz8wsICEhJSenp6TEPMjg4mJeXt2nTJg8Pj6CgoPT09I6ODttPc0ExMTGEkB9//HGF4gOsOUiY1jllwtTr9YcPH66trV3sZjoKCROAfyu+myoArDDsP+9qCIf93vv7+yMjI4ODgxsbG0dGRm7evJmens4wTHV1tamOSCR65plnzJ8VFxdH10uzUoeSSqUikSghIeH69evj4+NarfbJJ5/08PC4evWqXeInJSWtX79eo9FYP03Whvf/119/TQj58MMPLcq1Wq1EIqF/Dw4ORkREEELOnj1LS0xLyplw6ep9+/YRQvbt20e7q7m5WSgU7tixw1TBYDA8+uijwcHBly5dGhsbu3HjRmJiopeX1/Xr15dxaizLhoeHr1u3brGjIyMjhBCZTLaMyBkZGRkZGctr1UrEAbCCY35wnYS57M+dUyZMhULx5ptvmp/g8ePH51ezJWFyuV4DwJIwzwIAwAkVFRXdvn37k08+SUlJEYvFW7ZsOXfuXGhoaF5e3mI7SjysiYmJM2fOJCQkiESi+Pj4s2fPTk9PHzp0yC7B5+bm6FXKLtEW1N/fTwiRSCRW6gQGBqrVaoFAkJub29XVtWAd7l2dnZ1Nu+vZZ59NTk7WarVGo9EU5M6dOx9//PHzzz/v4+MTHR19/vx5lmWX/MVyecRiMcMwtAcAAAlzSc6XMKurq2/dunXq1KklayJhAvAOYxYAAE6ovr6eEJKcnGwq8fT03Lt37+TkpL0muIpEIjpjltq2bVtYWJhOp7PLF7urV6/ev38/ISHB9lCLoTddCwQC69V27txZXl4+MTGhVConJyfnV+De1Tt27DD9TX+NNBgM9GFDQ4Obm1tKSoqpQkhISHR0dHt7u16vf9hT48Ld3X3B0wFwQUiYS3KyhHn37t133323trZWJBJxqY+ECcAvjFkAADibBw8ejIyMeHl5WdyjGxwcTAgZGBiwy6v4+flZlGzYsIEQ8vfff9sl/krz8vIihMzMzCxZMy8vLzMz88aNGxY7/JGH7Grznyg9PDwIIXNzc6Ygc3NzEomEMfPLL78QQm7durW8E7RudnaW44KdAM4NCZMLJ0uY9M6U3bt3m55O9zotKSmhD//44w/z+kiYAPzCmAUAgLPx9PSUSCRTU1NjY2Pm5XTmbUhICH3o5uY2PT1tXmF4eNgiFMMwi73K0NCQxVRk+uWbfhG3Pf5KCw0NJYTQG5WXVFNTs3Xr1traWnrPswnHrrbO09PTz8/P3d19ZmZm/j2cSUlJXE+Js9HRUZZlaQ8AuDgkTC6cLGG+9dZbFk+0WM/iscceM1VGwgTgHcYsAACcUFpaGiHEfG+2Bw8etLS0CIVChUJBS0JDQ/v6+kwVBgYG7t69axHH29vb9DV669atn3/+uenQ1NSUVqs1Pfztt98MBoNUKjV9sbMx/kp74oknCCEcJxL7+Ph89913IpHozJkzFoe4dPWS0tPTZ2dnTfsIUCdPnnzkkUdmZ2c5BuGO/l9oDwAAEuaSkDCRMAF4hDELAAAnVFZWFhkZqVKpmpqaxsbGuru7X3nllf7+/tOnT9NZuIQQuVxuMBg+/fTT8fHxnp6eQ4cOmX7xM9m+fXt3d/dff/2l0Wh6e3tlMpnpkEQiOXLkiEajmZiYaGtry8rK8vDwOH36tKmCLfH37NkTEBDw888/279r/iOVSjds2KDT6TjWj46Orqqqml/OpauXVFZWFhUV9frrr1++fHlkZOT+/ftVVVUffPBBeXm5aUfDrKwshmFu377NMaYVdFNAuVxueygAJ4CEuSQkTCRMAD7ZYe8RAOAV9jp1NYTb3mlGo1GlUkVGRgoEAolEolAoWlpazCsMDw9nZ2eHhoYKhcJdu3ZpLNUingAAAvNJREFUtdq4uDh6aXjvvfdona6uLplMJhKJIiIiKioqTM+VSqXh4eGdnZ0KhcLX11coFCYmJra2ttorvkwm8/f357JxnS3v/yNHjri7u/f19dGHg4OD5tfHuLi4+U85ePCgxdZ9rNWu1mg05jGPHj3K/n9+eHJyMq05NDSUn5+/efNmgUAQFBQkl8ubm5vNX2XPnj0+Pj6zs7NWzqixsXH+hd58E0FKqVSGh4dPT09z7Skz2OsU1hDu+cFFEqYtnzvnS5hUbm6uRc5UKBTmFWxJmByv1wBgHcOu5MZIALAK1Gp1ZmYmPsuug2GYurq6/fv389iGmJgYo9G4QltaPBRb3v8jIyPR0dEpKSmVlZV2b5h9DQ8Ph4WFHThwoLq62sZQOp0uNjb23LlzL7/88jKerlQqCSEXLlywsRn2igNghYNcHx0nYdryuUPCXMbTHeF6DeAEcG8IAAC4KIlE0tjYePHixYqKCr7bYg3Lsnl5eWKx+Pjx4zaG6u3tTU9PLyoqWt73bwBwWUiYAMAXjFkAAIDrio2NbWtru3z58ujoKN9tWdS9e/d6e3tbWlo4rqtvRVVVVWlpaWlpqV0aBgAuBQkTAHiBMQsAAHgI5eXlDMPodLq+vj6GYYqLi/luka02bdrU1NQkFov5bsiiQkJCWltbo6OjbQ918uRJ/GAIsGqQMFcfEiaA83HnuwEAALCWFBQUFBQU8N0KAIA1AAkTAMB2mGcBAAAAAAAAAI4IYxYAAAAAAAAA4IgwZgEAAAAAAAAAjghjFgAAAAAAAADgiLAGJ4CTUKvVfDcBVo9Go+G7CY6CdgXe/6tGr9dv3LjRXqHwj4MVhfxgQa/XE3QIAKw1DMuyfLcBAGyiVqszMzP5bgUAuIqMjIwLFy7YGESpVF68eNEu7QEAcFh1dXX79+/nuxUAaxvGLAAAAAAAAADAEWE9CwAAAAAAAABwRBizAAAAAAAAAABHhDELAAAAAAAAAHBEGLMAAAAAAAAAAEf0L4FndfQNVbSDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 766 ms (started: 2021-07-27 16:31:02 +08:00)\n"
     ]
    }
   ],
   "source": [
    "keras.utils.plot_model(model, \"ticket_classifier_with_shape_info.png\", show_shapes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rolled-creativity",
   "metadata": {},
   "source": [
    "张量形状中的“None”表示批次大小：此模型允许任何大小的批次。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daily-present",
   "metadata": {},
   "source": [
    "> 使用函数模型进行特征提取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "automated-place",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tensorflow.python.keras.engine.input_layer.InputLayer at 0x7fc14022d4f0>,\n",
       " <tensorflow.python.keras.engine.input_layer.InputLayer at 0x7fc14022d160>,\n",
       " <tensorflow.python.keras.engine.input_layer.InputLayer at 0x7fc14022da60>,\n",
       " <tensorflow.python.keras.layers.merge.Concatenate at 0x7fc14022dc40>,\n",
       " <tensorflow.python.keras.layers.core.Dense at 0x7fc1041ba2b0>,\n",
       " <tensorflow.python.keras.layers.core.Dense at 0x7fc1041ba880>,\n",
       " <tensorflow.python.keras.layers.core.Dense at 0x7fc1283fb220>]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.24 ms (started: 2021-07-27 16:32:13 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "conventional-funeral",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<KerasTensor: shape=(None, 10000) dtype=float32 (created by layer 'title')>,\n",
       " <KerasTensor: shape=(None, 10000) dtype=float32 (created by layer 'text_body')>,\n",
       " <KerasTensor: shape=(None, 100) dtype=float32 (created by layer 'tags')>]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.97 ms (started: 2021-07-27 16:32:29 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.layers[3].input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "subtle-tower",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<KerasTensor: shape=(None, 20100) dtype=float32 (created by layer 'concatenate')>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.84 ms (started: 2021-07-27 16:33:04 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.layers[3].output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abandoned-nutrition",
   "metadata": {},
   "source": [
    "这使您能够进行特征提取：创建重用另一个模型的中间特征的模型。 假设您想向我们上面定义的模型添加另一个输出——您还想预测一个给定的问题单需要多长时间来解决，一种难度等级。 您可以通过超过 3 个类别的分类层来做到这一点——“快速”、“中等”、“困难”。 您无需从头开始重新创建和重新训练模型！ 您可以从之前模型的中间特征开始，因为您可以访问它们。 像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "lonely-excitement",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 26.7 ms (started: 2021-07-27 16:35:19 +08:00)\n"
     ]
    }
   ],
   "source": [
    "features = model.layers[4].output\n",
    "difficulty = layers.Dense(3, activation=\"softmax\", name=\"difficulty\")(features)\n",
    "\n",
    "new_model = keras.Model(\n",
    "    inputs=[title, text_body, tags],\n",
    "    outputs=[priority, department, difficulty])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "adaptive-diversity",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABZEAAAGVCAIAAAAACyiUAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzde1wTV94/8JMQLiGEoIAQASvYQleq0QIiCouAC1qoFx6QWu1tF1+s3RaptSuobZ9Xq7W11MputVVp9+mrYoXaR1+rVVsXb+WiBhSsVdRFVBCCXOQqt8D8/ji/zjObQAgQMgE+77/IycnMmZnM9wzfnDkjYBiGAAAAAAAAAACYGCHfDQAAAAAAAAAA6AVyFgAAAAAAAABgipCzAAAAAAAAAABThJwFAAAAAAAAAJgiEfdFfn7+9u3b+WoKAMBI9N133w19Idu3b8/Pzx/6cgAAwHSsXbs2ICBgiAvB9TkAjDUBAQFr165lX/7HOIvy8vKDBw8avUkwAlRUVOC7wXX+/Pnz58/z3QrgmQHPi/z8fHyjYOQ6ePBgRUUF360wFegxgTp48GB5efnQl4Prc+gXrks1oFca0c6fP6/xS55Iu5JBfjOEUSYrKysuLg7fDVZsbCzByTLm0fPCUEubPXs2vlEwQgkEgjfeeGPZsmV8N8QkoMcESiAQGHBp+EaBDrgu1YBeaUSj32cuzGcBAAAAAAAAAKYIOQsAAAAAAAAAMEXIWQAAAAAAAACAKULOAgAAAAAAAABMEXIWAMNo3759gt/Y2NhovHv37t1FixY1NTXV1tay1WbOnNne3s6txn1XIBD4+voacQsG5tixY56eniJRL5P7UkVFRZGRkXZ2dlKpdP78+bm5uXzVSU5OzszM1PhIcnIyu59nz56t1zYDAAwKOggN6CAAwJgQhDWYcvBEzgKGV0tLyxNPPBEVFcV3Q/j0+eefMwzT0tLCLSwqKvL19Q0PD7e1tXVwcGAYRqlU0vKkpCRuTfpufn6+vb09wzAFBQVGbb1+SktLFy1alJKSUl1d3VedCxcuzJkzRyqVXr9+vayszMPDY968eT/99BMvdVatWpWSkvL2229zP/Xhhx8yDMMwjJmZ2ZB2BwDoAR0EQQfxG3QQALxAHEYQpkw9eDIcNC/CAGgZ9HejqanJw8Nj4cKFBm+SniQSydy5cw2+2JiYmJiYmH6rffPNN+S3aMjV2Njo6uqakJDALVQqlZaWlvb29oSQ/fv3a3yEjYamafny5Vu3bu3q6nJxcTEzM9Ou0N3d7e3tLZfLHz16REvUarWXl5ebm1t7e7vx6zAMU1RUJBAIMjMztVtrZmbm7+/f71YbMGbq+Y0CME2EkF5PJd1GawehZ2RAB8EalR0EM9jzQhuuz6Ffg76KGK1xWJ+zD0GYZWrBU/v7jHEWMLykUmlpaemxY8f4bohp2bZtm0qleueddzTKraysMjIyhEJhQkLCzZs3eWnb4Hz55ZfJyck6xpudO3fu119/jYmJEYvFtMTMzGz58uXl5eVHjx41fh1CiEKhiImJefPNN9VqtcF2BADoDR1Er9BBEHQQAMaCOKwNQZiYXvBEzgLA2BiGSU9P9/f3nzhxova7ERERmzZtam5ujo2N1bhlzpSx8asvp06dIoRo3OZHX2ZnZxu/DrV06dKKiooffvihv+0DADAGdBAsdBAAYHwIwiyTCp7IWcAwOnz4MDvnCj2xuSV37tyJi4uzs7Ozt7ePiooqLS2ln0pNTaUVXF1dlUplWFiYVCq1trYOCQlhJ3rZvHkzrRMYGEhLTpw4QUscHBy4y2ltbc3NzaVv6cgvGlNxcXF1dbVCoeirwrvvvhseHn7lypXXX39d96Lq6urWrl07ZcoUCwuLcePGLVy48PTp0/QtfXY1VVNTk5iYOHnyZAsLC0dHx+jo6KKioqFvpoaSkhJCiKurK7fQxcWFEMLmqo1Zh5oxYwYh5McffxzClgHAYKCD6BU6CBY6CIDhhjisDUGYZVLBEzkLGEZLlixhGGbx4sW9liQlJSUlJd2/fz8zM/PUqVPLly+nddatW8cwjEKhaGhoWLNmzebNm1Uq1blz5+rr60NDQ8+ePUsI2bRpE8MwEomEXfKCBQsYhvHx8WFL6HK4t8lxBymFhoba29ufP39+uHeCtqtXrxKtc55LKBRmZGS4ubmlp6dnZGT0VU2lUvn5+e3fvz8tLa22tvbChQvW1tZhYWHp6elEv11NCKmqqvLz88vKytq1a1d9ff2ZM2fq6+sDAgLy8/MNuc2ENDQ0EEK4h4wQQmdpfvjwofHrUDTU0iMCAMaEDqJX6CBY6CAAhhvisDYEYZZJBU/kLIA38fHxAQEBEolk/vz5kZGRSqWytraWW6G1tXXXrl20jq+v7759+zo7O9esWWOQtff09ND4aJClDUhVVRUhRCaT6ajj4OCQlZVlbm6ekJBA85raUlJSysrKduzYERUVZWtr6+npuX//frlcnpiYqDEtsI5dnZKScvfu3e3btz/zzDM2Njbe3t4HDhxgGKbf5LFB0P0vEAj4qmNraysQCOgRAQDTgQ5CRx10EMapgw4CxrixGYcRhFkmFTyRswDe+Pn5sX+7ubkRQiorK7kVJBIJHVxETZs2beLEicXFxQY5B9hU5dAXNVB09J25ubnuarNnz05NTW1tbY2NjW1ra9OucOjQIUJIZGQkW2JpaRkWFtbW1qYxHEvHrj58+LBQKOQ+48rZ2dnb27uwsLCiomKgm6aDnZ0dIaS1tZVbSF/St4xchyUSiXrdvQDAI3QQuquhgxjWOix0EDCWjc04jCDMMqngiZwF8IabwrSwsCCE9PT0cCtonwATJkwghDx48GD4WzeMrKysCCFdXV391kxMTIyLi7t69eprr72m8VZHR0djY6OVlZVUKuWWOzk5EUJUKhW3sK9dTRfS09Mjk8kEHJcuXSKE3Lp1a3Ab2Ksnn3ySEKIRYe/fv08I8fT0NH4dllqt7ndqIgAwMnQQ/dZEBzF8dVjoIGAsG5txGEGYZVLBEzkLMF11dXUaQ8JoEKQBkRAiFAo7Ozu5FeitVly6RzTxQi6XE0IaGxv1qZyenu7l5fXVV1/Rh0izLC0tZTJZe3t7c3Mzt5yON3N2dtZn4ZaWlnZ2diKRqKurS/vZyCEhIfpukh7o0goLC7mF9GVYWJjx61BNTU0Mw9AjAgAjCDoIgg4CHQQAr0ZlHEYQZplU8ETOAkxXe3u7UqlkX/7yyy+VlZUKhYI9B+RyOU3vUSqV6t69exoLsba2ZsOll5fXnj17hrnV/XvqqaeIVp6yLzY2Nt9//71EItm1a5fGW0uXLiWEcJ8k1NHRkZ2dLRaLIyIi9GxMdHS0Wq1m53mmPvroo0mTJhn2ucrBwcFTp049ePAg+2io7u7uAwcOuLm5sQPnjFmHot8fekQAYARBB0HQQaCDAODVqIzDCMK0xNSCJ3IWYLpkMtmGDRvy8/NbW1sLCgpWrlxpYWGRlpbGVggPD6+srPzss89aWlpKS0vXrFnDZnZZTz/99M2bN8vLy/Pz82/fvh0UFETLeZwWXqFQTJgwobi4WM/63t7eu3fv1i7funWru7t7UlLS0aNHm5ubb968+fzzz1dVVaWlpdGxZ/rYunXrlClT/vjHPx4/fryxsbG+vn737t3vvfdeamoq+8SplStXCgSCsrIyPZfZK6FQ+OWXX9bX17/yyisqlaquru4vf/nLrVu39u7dS4fhGbkORZ8XFR4ePpRNAwDjQwdBoYNABwHAl1EZhxGETTR4cgeZZGZmapQAUIP7btDpZ1grVqzQeDbPxo0bmf8cVBYZGUk/q1AoXFxcrl27FhERIZVKxWJxcHBwTk4Od/kNDQ3x8fFyuVwsFgcGBiqVSvYRSuvXr6d1SkpKgoKCJBKJm5vbzp072c8GBQWNGzcuLy9vcDskJiYmJiam32p0qNjnn3+uUb5hwwaRSHT//n36sqamhrsTfHx8tBe1evVqe3t7jcLa2tqkpCR3d3dzc3OZTBYREZGdnU3f0n9X08dHe3h4mJubOzo6hoeHnzx5kruW0NBQGxsbtVqtY0uPHDmiHV727t2rUe3SpUsLFy60tbW1sbEJDQ3VOKDGrxMbG+vi4tLZ2alRbmZm5u/vr2N7KQPGTD2/UQCmiRCSmZk5oI+M4g5Cz8iADkKj2ijrIJhBnRe9wvU59GtwVxGjOA7rc/YhCGtUM53gqf19Rs4C9GL87wYNhcZc44AMMWfR0NDg4uKSkJAwPK0zpIcPH4rF4vj4eL4bYnhFRUUCgeDbb7/Vfgs5C4ABMdT/Znoy8Q5iiDkLdBCmYOgdBIOcBRiR8a8iTDwODyVngSA8FAYJntrfZ9wbAsADmUx25MiRgwcP7ty5k++26MIwTGJioq2t7fvvv893Wwzs9u3b0dHRKSkpzz33HN9tAQD4P+ggeIcOAmAsQxAetOELnjzkLFJTU+mTWlxdXQ27ZBsbG+6TYFJTUw27/KEw5bbBcFu9erVAILCxseEWzpw5s6Cg4Pjx401NTXw1rF/V1dW3b9/Ozs7Wc4rjEWT37t1btmzZsmULtzA5OZmeod3d3Xw1bAQZpmB+4MABuliN2yOHzpTjsCm3DYYVOggThA5iOJhylDPltsFwQxA2oGEMntxBF0Mce9bc3Pz444+zt9/oNkwDii5fvkwIWbx4scGXPHSm3LZ+GXNc4scff8z9itJbvEwNRvIDg3tDGIYZtmAeFhZmaWlp8MWachw25bb1ixjr3pAR0UFgJD9QhjovRs03ypSjnCm3TR/GvIoYEXHYaL0SDAeD3RtiY2MTGBionf7o6enp6enpt+boNgY32bDWrVvH/Y5u3ryZ7xYBmJDhizCIXcaB/TwU6CAAdEB46Rd20dAhDoPxiQy4LKlUWlpaasAFAgAAAAAAAMCYhTk4AQAAAAAAAMAUDThnQSdda21tzc3NpdNpiEQiQsjhw4fZqWva29t11OxLTU1NYmLi5MmTLSwsHB0do6Oji4qKBr1hLG7D7ty5ExcXZ2dnZ29vHxUVxY4K4c4kp1Qqw8LCpFKptbV1SEhIbm4urbN582Zahx1RduLECVri4OCge+foT61WZ2Zm/uEPf3B2dhaLxdOmTUtLS6O32zQ0NHDnB6IDsdRqNVsSExNDF6JjT3L3xo0bN5YtW2Zvb09f1tbWDmlHA8Aw0x1hdJz4gYGB7Im/cuVKQsj8+fPZkoaGhqHHrpKSksjISJlMphE5Kfp08SlTplhYWIwbN27hwoWnT5/W+PiSJUtkMplEIgkKCsrJyWHf0j/0DQ76CPQRAKOAjvCiI26w2CBsbW09a9aso0ePst1EfHw8IaSjo+Odd9558sknra2tx48f/+yzz/7zn/8c+nSkiMCIwAB64d6PpP8cPxKJZO7cudrlixcvJoS0tbX1W1Nj2rbKysrHHnvMycnphx9+aG5uvnr1anBwsJWVVV5eHlsnJCRk/Pjx+fn5OhrW1ww6tGGLFy/Oy8traWk5efKkWCz28/PTaJJEIgkICKB1lErl9OnTLSwszpw5o2NzfHx87O3t9dk5+szuc+TIEULIBx98UF9fX1NT87e//U0oFHJvG4uIiBAKhf/+97+5nwoICMjIyKB/67Mn6d4IDg4+ffp0a2vr+fPnzczMampqdDRs1Mz/ZCgjd8ZEMCBe5uDsNcL0e+IXFRVJJBKFQtHS0sIwTHt7u7+/v8bTs/uKXbopFAqZTBYSEpKTk9Pc3KwdOauqqtzd3Z2cnI4cOdLY2Hjjxo3o6GiBQLB3715a4datW3Z2di4uLj/99FNzc/OVK1fCw8MnT57MnYOz39DHoI9gGIa/PoJgtjMO9JhAGeq8GOL1eb9xQyMIX716df78+Y6OjtwgHB8fL5PJfvrpp0ePHqlUqnXr1hFCTp8+zVZABGb4i8AMrku1oFca0bS/z6aSs3jppZcIIdyrz6qqKktLSx8fH7YkODh43Lhx3LNam+5oeOTIEbaE5ju5IUChUBBCLl++zJZcuXKFEKJQKHRsjsGj4bx587glK1euNDc3b2xspC9//PFHQsirr77KVsjJyXFxcens7KQv9dmTdG8cO3ZMR0s04ApMA/oGYEwpZ6HPiZ+VlUUIiY6O7unpeemllzZs2KDPkvtFIyf3OlUjcr788suEEG5+pL29feLEiWKxWKVSMQwTGxtLCDl48CBb4f79+5aWltzL5X5DH4M+gmEY/voIXB1yoccEynRyFrrjhnYQfvDggbW1NTcIu7u7z5kzh7sQT09Pbs4CEZjhLwIzuC7Vgl5pRDPdnIVMJhMKhew5Tz399NOEkPLycn2aROmOhvT6mHrjjTcIIcXFxdwmSSQSjQ9OnDiREFJZWdnX5hg2GmqjzxPi9gHTpk2ztraura1lN+3DDz9k39VnT9K9wS5BH/S7AQDaBnRG92WIOQs9Q+jGjRsJIXPmzImKiuru7tZnyf1SKBRWVlY9PT3cQm7klMlkhJCmpiZuhRdeeIEQ8vXXXzMMI5VKCSHNzc3cCtOmTdN41qnu0Kcn9BHD1EfwceYBjACmkLPQphE3eg3CTz/9NDcIr169mhCyatWq/Px8tVo9kOb/H0TgYYrAzG9ZHoBRQ+Oq2JDPDRm0jo6OxsZGQgi9tNVw69YtV1dXg6yIu3wLCwtCiMbtfHZ2dhofmTBhQmVl5YMHD+RyuUHaoFtjY+Mnn3xy6NChioqKhoYGtvzRo0fs30lJSX/605927dr19ttv37x589SpU//4xz/oWwPakxKJZKDNQ+aC9emnnxJCaJ8KY1Z+fv6OHTv4bsUATvz333//X//6V15e3tdffy0UGmwOZnrDLbeEjZzjx49vbGy0srKi18QsJycnQohKpero6GhubraysrKxsdFYws2bN7klOkKfoaCPGEofkZSUFBAQMKCPjFY0MqDHhLi4OL6bQEh/caOvIDxu3Djuy507dwYEBHz99ddhYWGEkKCgoISEhKVLlxqwnYjAQ7xKnz17Nq5LWXFxceiVRi76fxbXIHMWGpenQ6xpaWlpZ2fX0tLS1tY20OlwDKuuro5hGG6bHzx4QAiZMGECfSkUCjs7O7kf4cYsSv+do+3ZZ5/9+eef09LSli9f7uDgIBAIduzY8cYbbzCcX7FWrFixYcOGzz777K9//esnn3zy0ksvsf3KcO/JZcuWGXyZI9R3331HsEOAEOPnLLQjjP4n/pkzZxobG6dNm/bqq68qFAo61FbHkvVEL8K42MhpaWkpk8kaGxubm5u5aYvq6mpCiLOzs6WlpVQqbW5ubmlp4V4x19fXayxTR+gzGvQROgQEBCAksnbs2IG9AcbPWfQaXnTHjb6CMA1u3CW/8MILL7zwQldX15kzZ1JTU6Ojoz/55JO1a9cO90axEIF1c3V1RdhhxcXFoVcauej/WVyD/J3N2tqaDQpeXl579uwZYs3o6Gi1Wq0x1fxHH300adIktVo9uEYOQnt7u1KpZF/+8ssvlZWVCoWCTd/K5fL79++zFVQq1b179zQWov/OYYlEopKSku7u7tzcXGdn58TEREdHRxpV29raNCpbWlq++uqrDx48+OSTTzIyMtasWcN910T2JAAMk14jjD4nfllZ2Z/+9Kfvv//+n//8p1gsXrx4cU1NTb9L1kdLS0txcTH7UiNy0h/ifvjhB7ZCR0dHdna2WCyOiIgghCxcuJAQcuLECbZCbW3tjRs3NNaiO/QZB/oIADBl2uFFn7ihHYRVKpXGSDc7O7uSkhJCiLm5+R/+8Af6hAtuYDcCRGCAMWuQOYunn3765s2b5eXl+fn5t2/fDgoKGmLNrVu3Tpky5Y9//OPx48cbGxvr6+t379793nvvpaamspnI0NBQe3v78+fPD67N+pDJZBs2bMjPz29tbS0oKFi5cqWFhUVaWhpbITw8vLKy8rPPPmtpaSktLV2zZg2b3GXpv3M0mJmZzZs3T6VSffzxx7W1tW1tbadPn/7iiy+0a7766qtisXjTpk3z589//PHHuW/psycBYOTqNcL0e+K3tLQsWbJkx44dU6dOnTx58sGDBysrK2NiYrq6unQvWR8SieS11167cOFCr5Fz69at7u7uSUlJR48ebW5uvnnz5vPPP19VVZWWlkbvEPnggw/Gjx+flJR08uTJlpaWa9eurVy5UmOUMqUj9BH0ERzoIwDGJu3wok/c0AjCV69efeWVV5ydnTUW/uc///nKlSsdHR0PHjzYtm0bwzChoaHsu4jALERgAMPjTm6h/xw/JSUlQUFBEonEzc1t586dDMMcOnSIu9gVK1b0VZNOV8PauHEjrVlXV7d27VoPDw9zc3NHR8fw8PCTJ09yVxoUFKR7RmKNW78+/vhjhmHy8/O1V8ctiYyMpB+n04Jeu3YtIiJCKpWKxeLg4OCcnBzuKhoaGuLj4+VyuVgsDgwMVCqVPj4+dDnr16/va5O126bt+vXrDMPU1NQkJCS4ubmZm5s7OTm9/PLLycnJtAJ3SmGGYVatWkUIOXv2rPZ+0LEnNfaGnoebwSzoWjA/MzA8PTek1wjD6Dzx//KXv7Cn/C+//KIxvOL999/XveS+sMHcxcXl4sWLISEhNjY2vUbO2trapKQkd3d3c3NzmUwWERGRnZ3NrXDjxo0lS5bY2trS59sdPXqU3jJNCPnTn/7Erakj9KGP0HNHDUcfQTBDOwd6TKAMdV4M5fqc0S9usEHY2tp6zpw5Z8+enTdvnrW1NbvkoqKihISE3/3ud9bW1uPHj589e/bevXu5sy8jAnNbYuQIzOC6VAt6pRFN+/ssYDihISsrKy4ujhmrE4DPmDGjtra2oqKC74bo5R//+MfOnTsLCgqMs7ox/t3QRh8Mpn23FYwpBjwv8I3Sk5FDHxf6CB0EAkFmZibuHKbQYwJlqPOCl2/Uk08+2dbWdvfuXWOuVAdEYN1wFaEBvdKIpv19Nti88WBkX3zxhTHnPYLB2bdvn+A32mPd7969u2jRoqamptraWrbazJkz29vbudW47woEAl9fXyNuwcAcO3bM09NTx/jGoqKiyMhIOzs7qVQ6f/58jVs6jVknOTlZe1b/5ORkdj/Pnj1br20G40Lo0xN2lOlDB6EBHQRfVCrV+PHjubcK3rlzp7S0lHvrBwwIIvCIgCCswZSDJ3IWI0l6evrSpUtbWlq++OKLhw8fInc4Unz++ecMw7S0tHALi4qKfH19w8PDbW1tHRwcGIahM0sVFRUlJSVxa9J38/Pz6TPGefmFuV+lpaWLFi1KSUmhz4Po1YULF+bMmSOVSq9fv15WVubh4TFv3ryffvqJlzqrVq1KSUl5++23uZ9iH6JuZmY2pN0BBoXQpyfsqJEIHQSFDoJfDx8+TEhIKC8vf/To0cWLF+Pi4mxtbTX2AOiGCDxCIQhTph48uTeKjNk7MPuaYsPU7N27lxAiEommT59eWFhozFUb+bshkUjmzp1rysvX877Bb775hvwWDbkaGxtdXV0TEhK4hUql0tLS0t7enhCyf/9+jY+w0dA0LV++fOvWrV1dXS4uLmZmZtoVuru7vb295XL5o0ePaIlarfby8nJzc2tvbzd+HYZhioqK6NBB7daamZn5+/v3u9W8zGdhTDr6jnfffddozeAx9FHoI/pFjHjnsOl3EHpGBnQQrFHZQTB8zGcxaP/617+WLl06efJkCwsLJyenFStW/Pvf/x7WNeoPEVgfRr6KMP04rM/ZhyDMMrXgqf19xjgLQghZt24dd6ds3ryZ7xb1Lj4+nmGYrq6u4uLip59+mu/mwOBt27ZNpVK98847GuVWVlYZGRlCoTAhIUHjMWMm7ssvv0xOTtYx3uzcuXO//vprTEyMWCymJWZmZsuXLy8vLz969Kjx6xBCFApFTEzMm2++iQeM9UVHd/Lf//3fRmsG76EPfQQYEzoIgg7C6MLCwv73f/+3rKyso6NDpVLt27dvypQpfDfq/0MEBiNDECamFzyRswAwNoZh0tPT/f39J06cqP1uRETEpk2bmpubY2NjNW6ZM2Vs/OrLqVOnCCEat/nRl9nZ2cavQy1durSiosLIT5gHAOgLOggWOggAMD4EYZZJBU/kLMDA6DOcpkyZYmFhMW7cuIULF54+fZq+tXnzZjr/SmBgIC05ceIELXFwcKAlqampAoGgtbU1NzeXvkWTgrRcIBC4uroqlcqwsDCpVGptbR0SEsLO/jKU5RtTcXFxdXW1QqHoq8K7774bHh5+5cqV119/XfeidOztw4cPsxPe3LlzJy4uzs7Ozt7ePioqqrS0lLuQmpqaxMREOiLU0dExOjq6qKho6JupoaSkhBDi6urKLXRxcSGEsLlqY9ahZsyYQQj58ccfh7BlAKAvdBD9QgfBQgcBMBwQh3VDEGaZVPBEzgIMSaVS+fn57d+/Py0trba29sKFC9bW1mFhYenp6YSQTZs2Mf/5EOwFCxYwDMM+PZv8NgKQexsbHVlEyxUKRUNDw5o1azZv3qxSqc6dO1dfXx8aGnr27NkhLp8KDQ21t7c/f/78cO4kcvXqVaJ1znMJhcKMjAw3N7f09PSMjIy+qune20uWLGEYZvHixYSQpKSkpKSk+/fvZ2Zmnjp1avny5exCqqqq/Pz8srKydu3aVV9ff+bMmfr6+oCAAO2HhA9RQ0MDIUTjEeh0luaHDx8avw5FQy09IgAwrNBB6AMdBAsdBIDBIQ73C0GYZVLBEzkLMKSUlJSysrIdO3ZERUXZ2tp6enru379fLpcnJibqmKh2QFpbW3ft2hUQECCRSHx9ffft29fZ2blmzRqDLLynp4fGR4MsrS9VVVWEEJlMpqOOg4NDVlaWubl5QkICzWtq039vx8fH0z02f/78yMhIpVJZW1vLLuTu3bvbt29/5plnbGxsvL29Dxw4wDBMv8ljg6C7WiAQ8FXH1tZWIBDQIwIAwwodhD7QQbDQQQAYHOJwvxCEWSYVPJGzAEM6dOgQISQyMpItsbS0DAsLa2trM9QAIYlEQkccUdOmTZs4cWJxcbFBTgw2fzn0RelA738zNzfXXW327Nmpqamtra2xsbFtbW3aFfTf235+fuzfbm5uhJDKykr68vDhw0KhMBZCFUkAACAASURBVCoqiq3g7Ozs7e1dWFhYUVEx0E3Twc7OjhDS2trKLaQv6VtGrsMSiUS97l4AMCx0EPpAB8FCBwFgcIjD/UIQZplU8ETOAgymo6OjsbHRyspKKpVyy52cnAghKpXKIGvRPismTJhACHnw4IFBlm8EVlZWhJCurq5+ayYmJsbFxV29evW1117TeGtAe5ubLbawsCCE9PT0sAvp6emRyWQCjkuXLhFCbt26NbgN7NWTTz5JCNGIsPfv3yeEeHp6Gr8OS61W9zs1EQAMEToIPaGDYKGDADAsxGF9IAizTCp4ImcBBmNpaSmTydrb25ubm7nldASUs7MzfSkUCjs7O7kV6P1RXDqGIdXV1WkMCaNBkAbEoS/fCORyOSGksbFRn8rp6eleXl5fffUVfYg0S8+9rZulpaWdnZ1IJOrq6tJ+mGVISIi+m6QHurTCwkJuIX0ZFhZm/DpUU1MTwzD0iADA8EEHoSd0ECx0EACGhTisDwRhlkkFT+QswJCWLl1KCOE+26ajoyM7O1ssFkdERNASuVxOc3KUSqW6d++exnKsra3ZcObl5bVnzx72rfb2dqVSyb785ZdfKisrFQoFe2IMcflG8NRTTxGtPGVfbGxsvv/+e4lEsmvXLo239Nnb/YqOjlar1eyUztRHH300adIkwz5XOTg4eOrUqQcPHmQfDdXd3X3gwAE3Nzd24Jwx61D0q0KPCAAMK3QQ+kAHQUvQQQAMB8ThfiEI0xKTC57chE1mZqZGCQCl53ejqqrK3d3dycnpyJEjTU1NN27ciI6OFggEe/bsYevQAVR///vfm5ub//3vfy9btszFxcXe3p67nAULFshksnv37uXl5YlEomvXrtFyhUIhk8nCwsLy8vJaWlqUSuX06dMtLCzOnDljkOWHhISMHz8+Pz+/3y2NiYmJiYnptxpNu37++efcwp6engkTJrDzIbOUSqVMJut1Ofv27SOEaGyFPnubzkjc1tbGlqxfv54QcvnyZfqyurp6ypQpHh4ex44da2hoqKur++KLL6ytrTMzM9mPrFixghBy+/btfreXYRgXFxczM7Ne38rPz7eysnruueeqqqpqa2sTEhJEItGJEyf4qsMwzP79+wkhhw4d0ig3MzPz9/fvd2MNGDP1/EYBmCZCCDdo9GrsdBB6RgZ0EFyjr4Ng9Dsv9IHrc+iXnlcRYycO63P2IQhzmVTw1P4+I2cBetH/u1FbW5uUlOTu7m5ubi6TySIiIrKzs7kVGhoa4uPj5XK5WCwODAxUKpXsI47Wr19P65SUlAQFBUkkEjc3t507d7KfVSgULi4u165di4iIkEqlYrE4ODg4JyfHUMsPCgoaN25cXl5ev5s5lJwFwzAbNmwQiUT379+nL2tqariZRB8fH+1FrV69WiMaMjr3tsZjkDZu3Mj851C9yMhIWpM+PtrDw8Pc3NzR0TE8PPzkyZPctYSGhtrY2KjVah1beuTIEe2U6N69ezWqXbp0aeHChba2tjY2NqGhoRrHzvh1YmNjXVxcOjs7NcqRswAYED3/NxsjHcRQchYMOojR0kEwyFmAEel/FTFG4rA+Zx+CsEY10wmeyFnAIJnId4OGQr5bwTBDzlk0NDS4uLgkJCQMT+sM6eHDh2KxOD4+nu+GGF5RUZFAIPj222+130LOAmBADPW/2VCYTgcxxJwFOghTMPQOgkHOAozIRK4iTCcODyVngSA8FAYJntrfZ8xnAcADmUx25MiRgwcP7ty5k++26MIwTGJioq2t7fvvv893Wwzs9u3b0dHRKSkpzz33HN9tAQD4P+ggeIcOAmAsQxAetOELnshZAAy71atXCwQCGxsbbuHMmTMLCgqOHz/e1NTEV8P6VV1dffv27ezsbD2nOB5Bdu/evWXLli1btnALk5OT6XOkuru7+WoYAIwp6CBMEDoIgLEDQdiAhi94ImcBI0NqaqpAICguLr5//75AINi0aRPfLdLLypUr2UFNLS0tGu9Onjz56NGjtra2vLRNH87Ozjk5Od7e3nw3xPA++ugj7Rzwhx9+yB6v8+fP89IwABgodBC8QAcBAKyRGIcRhA1u+IKnaMhtAzCGdevWrVu3ju9WAACAyUEHAQDAL8RhGFYYZwEAAAAAAAAApgg5CwAAAAAAAAAwRchZAAAAAAAAAIApQs4CAAAAAAAAAExRL3NwZmVlGb8dYOLy8/MJvhscFRUVBDuEEEJId3d3eXn5pEmThMIxlwOl54WhVFRU4BsFI5dhT4cRDT0mDIfR/Y3q6em5d+/eY489JhAI+G7LiITrUm3olUauiooKV1fX/yhiODIzM3lqGADASMUYQkxMDN/bAQAABpaZmTn0DgLX5wAw1sTExHDDoIBhGL6bBAAjWHd3d0lJSWFhYW5ubk5OzvXr1xmGkcvlgYGBc+fO9fHx8ff3Nzc357uZAGASsrKy4uLicO0BMDZVVlbSq4Xc3NzLly/39PSwFwyBgYEzZ84cg2M2AaBfyFkAgCE1NTVdvHgxJyeHZjEePnxoY2OjUCh8fHwCAwNDQkIcHBz4biMA8AY5C4AxRa1WFxcX06uCs2fP3rt3TyQSKRQKmqQIDg6eMGEC320EAFOHnAUADJdeh2B4eHjQ8Rf4RQVgDELOAmDUq66uvnjxItv7t7e3Ozk5+fn50a4/MDDQysqK7zYCwEiCnAUAGEljY6NSqaQ/tuTk5DQ0NNAhGHRQ6Jw5c+zt7fluIwAML+QsAEal27dv0zs+6E8UQqHQy8uLJinmzp07depUTK4JAIOGnAUA8ABDMADGJuQsAEaH1tbWy5cv0048Ly+vvr6e+zvE3Llzx48fz3cbAWCUQM4CAPjHDsHIzc3Ny8t79OiRVCqdPn06Ln0ARhnkLABGLnYGzcLCQqVS2dnZiSm3AcAIkLMAANOiVqtv3LjBXhVhCAbAaIKcBcAIwu2Rf/755zt37ohEIk9PT5qn+P3vfz958mS+2wgAox9yFgBg0lQqlVKppHeR5ObmtrW12drazpo1i01hjBs3ju82AoC+kLMAMHHs87/YkY9st0tTFWKxmO82AsDYgpwFAIwYGkMwrl27ZmZmhlm+AEYQ5CwATBCdQZP+PHDp0iV2eCP6VgAwBchZAMBIVVVVVVBQgCEYACMIchYApuDRo0eXLl2iHejp06dra2utra1nzpxJe8+QkBAHBwe+2wgA8P8hZwEAowF3CEZubu7t27cxBAPABCFnAcAXmuinHWVBQUFHR4dcLmd7ST8/P0tLS77bCADQC+QsAGAUqqysZB+kWlhY2N7eLpPJ/Pz86BCMoKAgOzs7vtsIMBYhZwFgNPSx4to3VLJP+vD29ua7jQAA/UPOAgBGObVaXVxcTK/YcnJyysrKMAQDgC/IWQAMq+bm5uLiYpqnyMnJaWhokEql/v7+SNkDwMiFnAUAjC3cIRh0cKyTk5Ofnx9NYcyZM8fa2prvNgKMWshZABhcZWUle2vk5cuXe3p65HI5TcrjAeEAMAogZwEAY1dXV9eVK1foEIxz587dvXuX++R5jJsFMDjkLACGjh0/mJube/bs2QcPHpibm0+fPp0mKYKDgydMmMB3GwEADAY5CwCA/097CIazs7Ovry97FwkeSg8wRMhZAAxOdXX1xYsX2U6qvb2d20MFBgZaWVnx3UYAgGGBnAUAQC/YIRi5ubnnzp2rrq7GEAyAoUPOAkB/t2/fpt1QTk7O9evXhUIhJmMCgDEIOQsAgP6xdwsXFhYqlcrOzk76iDgMwQAYEOQsAHRoaWkpKiqi3U1eXl59fb2NjY1CoaAdzdy5c8ePH893GwEAjA05CwCAgWltbb18+TIdoEtvJBaJRAqFgp2V3d3dne82Apgo5CwANHBz4hcvXuzq6mJn0PTx8fH39zc3N+e7jQAAfELOAgBgSPoagkGvOH19fXGPMQALOQsAtVp948YN2nH8/PPPd+7c4Sa+g4ODH3vsMb7bCABgQpCzAAAwGO4QjDNnztTU1LBzufv4+Pz+97+fPHky320E4BNyFjA2NTU1Xbx4kU5OkZub29bWZmtrO2vWLPqkD9xgCACgA3IWAADDhR2CkZube/ny5Z6eHu4QDD8/P0tLS77bCGBUyFnA2KExgybDMB4eHmySAjNoAgDoCTkLAABj4M6sdv78+draWu4QDAwGhjECOQsYxR49enTp0iU61O706dO1tbUSiWTGjBk0VR0SEuLg4MB3GwEARh7kLAAAeEB/f6OXthiCAWMHchYwylRVVRUUFNB8dEFBQUdHBzeYz5o1y8LCgu82AgCMbMhZAADwrLm5ubi4mF7y5ufn19XVWVtbz5w5k171BgcHT5gwge82AhgGchYw0nV3d5eUlLBTL1+7ds3MzMzLy4smKQIDAz08PPhuIwDAqIKcBQCAael1CAb73Dv8agcjGnIWMBI1NzdfuHCBRuacnJyGhgapVOrv78/OryyTyfhuIwDAqIWcBQCA6eIOwcjLy6uvr+feHT1v3jxHR0e+2wgwAMhZwEjR6yTK7GCKmTNnCoVCvtsIADAmIGcBADBicGehLykp0RiC4e/vb25uzncbAXRBzgJMVldX15UrV2iMPXv27IMHD9iZkpEjBgDgEXIWAAAjUlNT08WLF9m7SB4+fGhjY6NQKDAEA0wZchZgUqqrqy9evEijaE5OTnt7u7Ozs6+vL80F+/r6WllZ8d1GAICxDjkLAIARj84Jx152X79+nWEYDGMGE4ScBfBLO1oKhUIvLy/2SR9Tp04VCAR8NxMAAP4PchYAAKMNdwgGnS6ODsGgV+QBAQEODg58txHGKOQswPhaWlqKioq4EwNxQ2JgYOC4ceP4biMAAPQJOQsAgNGs1yEYHh4edAoMDMEAI0POAoyDnUGzsLDw4sWLXV1dmP0HAGCEQs4CAGAMaWxsVCqVdJK5vLy8R48eSaXS6dOn00v5OXPm2Nvb891GGM2Qs4Bholarb9y4QfMU586du3v3rkgkUigUNEkRHBz82GOP8d1GAAAYDOQsAADGKDoEg/0pEkMwwAiQswADYu+Dy83Nzc3NbWtrk8lkfn5+9I6PuXPnisVivtsIAABDhZwFAAAQQohKpVIqlfQuEnr1zx2CMXfu3PHjx/PdRhh5KioqXnrppe7ubvry4cOHZWVlTz/9NFvBy8tr9+7dPLUORh7uI5+5mVbMoAkAMFohZwEAAJq4o6wLCwuvXbtmZmaGqfVhcB5//PHS0tK+3n377bffe+89Y7YHRpZHjx5dunSJplNPnz5dW1srkUhmzJiBO9oAAMYI5CwAAKAfVVVVBQUF3CEYtra2s2bNYu8iwaz7oMN77723efPmrq6uXt/99ddfp06dauQmgYmrrKxkZw4uKCjo6OiQy+VsznTWrFkWFhZ8txEAAIwEOQsAABgADMGAgSotLX3iiSd6vd7w9va+evWq8ZsEpoY7vU5ubu7t27dpYKFRJSgoyN3dne82AgAAP5CzAACAwaNDMNgURnt7OzsHno+PT1BQkJ2dHd9tBP7NmDHjypUrGpcc5ubmW7Zseeutt/hqFfCrubn5woULNHTk5OQ0NDRIpVJ/f38aPX7/+9/LZDK+2wgAAPxDzgIAAAxDrVYXFxez/4GUlZVhCAZQ27dvX79+vVqt5hYKBILbt29PnjyZp0YBDyorK9nBFJcvX+7p6cGzigAAQDfkLAAAYFhw70jXGIIRGBg4Z84ca2vrgS6zq6vrxRdf3LZtm5ub23C0GYZJVVWVq6trT08PWyIUCv39/fPy8nhsFejvyJEjP//887Zt2wb6wa6uritXrtAkxZkzZ2pqaszNzadPn07jwLx58xwdHYejwQAAMGogZwEAAMOO/b+lsLDw559/vnPnjkgk8vT0pOMvfHx89ByCUVBQ4OfnJxaLt2zZ8vrrr4tEIiM0Hgzi97//fW5uLpu2EIlEf//73//85z/z2yro1927d1evXn38+HFnZ+eqqip9PkIfnMzNVzo7O/v6+tLz3c/Pz9LScribDQAAowZyFgAAYGzaDwWg/9Kwd5GIxeJeP/j3v/997dq1arVaKBT+7ne/++qrr2bNmmXkxsPg7N27989//jObszAzM6usrJwwYQK/rQId1Gr13/72t40bN3Z3d9PHvpSXl7u6umrXpDNosif19evXhUIh974wb29vozcfAABGCeQsAACAT9yh4+fOnauurtYYgsH9b+e55547ePBgd3c3IUQkEnV3d69YsSItLW38+PH8bQHo5eHDh05OTvRfXzMzs/nz5584cYLvRkGfioqKXnnllStXrrBpJqFQuH///ri4OPqypaWlqKiInZzi4cOHNjY2CoWCnrl4BDIAABgKchYAAGBC2Cn6CgsLlUplZ2enXC738fGhP9i+8MILKpWKW18kEslksu3bt7/44ot8tRn0FBkZ+eOPP3Z3dwuFwq+//nrlypV8twh68ejRo/fee+/jjz8WCoXcaVMtLCxWrFgRGhqal5eXm5v766+/dnd3T5kyZc6cOXPmzKGDKTCDJgAAGBxyFgAAYKJaW1svX75MB5yfPXv2wYMHvVYTCoUMwwQFBe3Zs8fLy8vIjQT9ffvttytWrGAYxtLSsra21sbGhu8WgaajR48mJCQ8ePBA4yEvlEwma21tVSgUdAzUvHnzJk2aZPxGAgDAmIKcBQAAjAw7d+58/fXX++q2RCKRUChMSUlJSUnBDH+mqbW11cHBob29PSYm5rvvvuO7OfAfqqqq1q1bt3//fqFQyH3CC5dQKKyurnZwcDBy2wAAYCzDED4AABgZ7t69a25u3te7arW6s7Nz8+bN06dPz8nJMWbDQE8SiWTJkiWEENwVYlJ6enp27tz5xBNP0ERSXwkL+tbVq1eN2DQAAACMswAYmfR5KiQAAAAAGArGiAHwAk+2BxipkpKSAgIC+G4FgJF0d3e/+OKL9B57MzMzgUDA3m9vZmZmZ2fn4ODg7Ozs6Ohob2/v4OBgb2/v6OhoYWFBCImLi8P5wsrPz9+xY0dmZiYva+/u7s7MzHz++ed5WTtoa25urqurq6urq6mpqa+vr6ure/DgQW1tbWNjI31AD/ntxiu1Wt3T06NQKDZs2MBvmwF48emnn/LdBIAxCuMsAEYkgUCQmZm5bNkyvhsCYCRKpXLhwoVyudzDw8Pd3f2xxx5zc3Nzc3ObNGmSXC7X/VmcL1xZWVlxcXE89v7t7e1WVlZ8rR30xDCMSqUqLy+vqKgoLy+/d+9eeXl5WVlZe3v7lStXMNYPxqDY2FhCCMZZABgfxlkAAMAI4OfnV1tby3crwACQsBgRBAKBXC6Xy+WzZs3iuy0AADCmYQ5OAAAAAAAAADBFyFkAAAAAAAAAgClCzgIAAAAAAAAATBFyFgAAAKBp3759gt/Y2NhovHv37t1FixY1NTXV1tay1WbOnNne3s6txn1XIBD4+voacQsG5tixY56eniJRn/N8FRUVRUZG2tnZSaXS+fPn5+bm8lUnOTl5iI99weHD4TMpRjh8ve725ORkdv/Mnj3bINsCAMMBOQsAAIBetLS0PPHEE1FRUXw3hE+ff/45wzAtLS3cwqKiIl9f3/DwcFtbWwcHB4ZhlEolLU9KSuLWpO/m5+fb29szDFNQUGDU1uuntLR00aJFKSkp1dXVfdW5cOHCnDlzpFLp9evXy8rKPDw85s2b99NPP/FSZ9WqVSkpKW+//fbgtheHD4fPdBjt8PW62z/88EOGYRiGMTMzM/imAYAhMQAwAhFCMjMz+W4FwMgwuPOlqanJw8Nj4cKFw9EkfUgkkrlz5xp8sfTHxn6rffPNN+S3nAVXY2Ojq6trQkICt1CpVFpaWtrb2xNC9u/fr/ER9r8m07R8+fKtW7d2dXW5uLiYmZlpV+ju7vb29pbL5Y8ePaIlarXay8vLzc2tvb3d+HUYhikqKqJP8B3oxuLwMTh8psRoh4/RudvNzMz8/f37bW1MTExMTMxAtxEAhg7jLAAAAHohlUpLS0uPHTvGd0NMy7Zt21Qq1TvvvKNRbmVllZGRIRQKExISbt68yUvbBufLL79MTk7WMS793Llzv/76a0xMjFgspiVmZmbLly8vLy8/evSo8esQQhQKRUxMzJtvvqlWqwe0sTh8BIfPlBjt8JEh7HYA4B1yFgAAAKAXhmHS09P9/f0nTpyo/W5ERMSmTZuam5tjY2M1bq03Zez/OX05deoUIURjOgD6Mjs72/h1qKVLl1ZUVPzwww/9bd//weFj4fCZCKMdPmoQux0ATAFyFgAAAJoOHz7Mzs1G/wHglty5cycuLs7Ozs7e3j4qKqq0tJR+KjU1lVZwdXVVKpVhYWFSqdTa2jokJISdEG7z5s20TmBgIC05ceIELXFwcOAup7W1NTc3l76l43dIYyouLq6urlYoFH1VePfdd8PDw69cufL666/rXlRdXd3atWunTJliYWExbty4hQsXnj59mr6lz66mampqEhMTJ0+ebGFh4ejoGB0dXVRUNPTN1FBSUkIIcXV15Ra6uLgQQtjftI1Zh5oxYwYh5Mcff9R/Q3D4WDh8Y+3wUYPY7QBgCpCzAAAA0LRkyRKGYRYvXtxrSVJSUlJS0v379zMzM0+dOrV8+XJaZ926dQzDKBSKhoaGNWvWbN68WaVSnTt3rr6+PjQ09OzZs4SQTZs2MQwjkUjYJS9YsIBhGB8fH7aELoc7nwV3MHNoaKi9vf358+eHeydou3r1KtH634BLKBRmZGS4ubmlp6dnZGT0VU2lUvn5+e3fvz8tLa22tvbChQvW1tZhYWHp6elEv11NCKmqqvLz88vKytq1a1d9ff2ZM2fq6+sDAgLy8/MNuc2ENDQ0EEK4h4wQQp+l8vDhQ+PXoei/ZPSI6AmHj4XDN9YOHzWI3Q4ApgA5CwAAgIGJj48PCAiQSCTz58+PjIxUKpW1tbXcCq2trbt27aJ1fH199+3b19nZuWbNGoOsvaenhyYyDLK0AamqqiKEyGQyHXUcHByysrLMzc0TEhLo75/aUlJSysrKduzYERUVZWtr6+npuX//frlcnpiYqPH4AB27OiUl5e7du9u3b3/mmWdsbGy8vb0PHDjAMEy/PzIbBN3/AoGArzq2trYCgYAeET3h8LFw+Mbm4RvEbgcAU4CcBQAAwMD4+fmxf7u5uRFCKisruRUkEgkdhExNmzZt4sSJxcXFBrlWZn/SHPqiBoreJmNubq672uzZs1NTU1tbW2NjY9va2rQrHDp0iBASGRnJllhaWoaFhbW1tWkM29axqw8fPiwUCrkPo3V2dvb29i4sLKyoqBjopulgZ2dHCGltbeUW0pf0LSPXYYlEol53b19w+Fg4fGPt8LEGutsBwBQgZwEAADAw3J86LSwsCCE9PT3cCtoXyhMmTCCEPHjwYPhbN4ysrKwIIV1dXf3WTExMjIuLu3r16muvvabxVkdHR2Njo5WVlVQq5ZY7OTkRQlQqFbewr11NF9LT0yOTyQQcly5dIoTcunVrcBvYqyeffJIQovGf2P379wkhnp6exq/DUqvV/U5hyIXDx8LhG2uHjzXQ3Q4ApgA5CwAAAAOrq6vTuHeDZito5oIQIhQKOzs7uRXoLdlcukc+80IulxNCGhsb9amcnp7u5eX11VdfffPNN9xyS0tLmUzW3t7e3NzMLafj0p2dnfVZuKWlpZ2dnUgk6urq0n6Qe0hIiL6bpAe6tMLCQm4hfRkWFmb8OlRTUxPDMPSI6AmHj4XDN9YOHzWI3Q4ApgA5CwAAAANrb29XKpXsy19++aWyslKhULDXynK5nP4MSKlUqnv37mksxNrams1reHl57dmzZ5hb3b+nnnqKaP2e2RcbG5vvv/9eIpHs2rVL462lS5cSQrhPHOzo6MjOzhaLxREREXo2Jjo6Wq1Wsw9koT766KNJkyZxpywduuDg4KlTpx48eJB9hGR3d/eBAwfc3NzYAfbGrEPR7w89InrC4aMlOHxk7B0+ahC7HQBMAXIWAAAABiaTyTZs2JCfn9/a2lpQULBy5UoLC4u0tDS2Qnh4eGVl5WeffdbS0lJaWrpmzRp2CAbr6aefvnnzZnl5eX5+/u3bt4OCgmg5j88NUSgUEyZMKC4u1rO+t7f37t27tcu3bt3q7u6elJR09OjR5ubmmzdvPv/881VVVWlpaXSMuj62bt06ZcqUP/7xj8ePH29sbKyvr9+9e/d7772XmprKPhp25cqVAoGgrKxMz2X2SigUfvnll/X19a+88opKpaqrq/vLX/5y69atvXv30uH6Rq5D0edKhoeHsyX9biwOHw4fGauHj9Le7QAwMmiP6QIA00cIyczM5LsVACPDIM4XOk0da8WKFRrP8Nu4cSPzn3d/REZG0s8qFAoXF5dr165FRERIpVKxWBwcHJyTk8NdfkNDQ3x8vFwuF4vFgYGBSqWSfdbp+vXraZ2SkpKgoCCJROLm5rZz5072s0FBQePGjcvLyxvc3sjMzNSn96dDyj///HON8g0bNohEovv379OXNTU13J3g4+OjvajVq1fb29trFNbW1iYlJbm7u5ubm8tksoiIiOzsbPqW/ru6rq5u7dq1Hh4e5ubmjo6O4eHhJ0+e5K4lNDTUxsZGrVbr2NIjR45oXx3t3btXo9qlS5cWLlxoa2trY2MTGhqqcUCNXyc2NtbFxaWzs3NAG4vDh8PHjOHDp73bKTMzM39/fx3tpGJiYmJiYvqtBgAGh5wFwIhEkLMA0JuRzxeaszDa6gZqiDmLhoYGFxeXhISE4WmdIT18+FAsFsfHx/PdEMMrKioSCATffvstW6LnxuLwmQIcPl5o73YWchYAJg73hgAADDulUvnyyy+7u7uLxeLx48c/9dRT//Vf//X555+Xlpby3TSAgZHJZEeOHDl48ODOnTv5bosuDMMkJiba2tq+//77fLfFwG7fvh0dHZ2SkvLcc8/RU/vhFQAAIABJREFUEv03FoePdzh8vNDe7QAwgiBnAQAwjHp6et566605c+ZMmDDh+PHjDQ0N169f//TTT5uaml599dXHH3/csNOVARjW6tWrBQKBjY0Nt3DmzJkFBQXHjx9vamriq2H9qq6uvn37dnZ2tp6PQhhBdu/evWXLli1btrAlA9pYHD5+4fDxQnu3E0KSk5PpU1q7u7v5ahgA6EPA/OddagAwIggEgszMzGXLlvHdEMOwsbGZMWNGTk7O6Fv7xo0bP/jggz179qxatYpb3t3d/eyzzx4/fryrq4udscxEjL7DYbTzJTU19a233mJfbty4cfPmzcO90oHKysqKi4tD7w8AMCCxsbGEkO+++47vhgCMORhnAQAwXEpKSj788EMfHx+NhAUhxMzM7O233+alVTB81q1bx7390gQTFgAAAAAji2n9uAcAMJrs2bOnp6eH/jKjLSAgAL91AwAAAADogHEWAKMZfRTZlClTLC0tXV1d58+f/z//8z9tbW3aFSwsLMaNG7dw4cLTp0/Ttw4fPiz4zZ07d+Li4uzs7Ozt7aOiojRmjtS9FrVanZmZ+Yc//MHZ2VksFk+bNi0tLa2np4e+m5qaKhAIWltbc3Nz6bq4N0rU1NQkJiZOnjzZwsLC0dExOjqaPlxd/+YN09r1dO7cOULI9OnT9ayPwzGshwMAAAAARh5jP6gEAAyB6PHsxqqqKnd3d2dn5yNHjjQ1NalUKjqD96effsqt4OTkdOTIkcbGxhs3bkRHRwsEAu6j0RcvXkwIWbx4cV5eXktLy8mTJ8VisZ+fn/5roU9f/+CDD+rr62tqav72t78JhUKNIfQSiWTu3Lka7a+srHzsscecnJx++OGH5ubmq1evBgcHW1lZ5eXl6d+8YV17SEjI+PHj8/Pz+zoEcrmcEHLhwoW+KnDhcAxx7Troc76MHXo+6xQAALjwrFMAvuCqBWBE0ud/sJdfflm72oIFC9h/X2kF7rPK29vbJ06cKBaLVSoVLaH/hR45coStExMTQwipqanRcy1HjhyZN28e992VK1eam5s3NjayJb3+m/rSSy8RQjIyMtiSqqoqS0tLHx8ftqTf5g3r2oODg8eNG6fj32aas7h48WJfFbhwOIa4dh2Qs+BCzgIAYBCQswDgC+azABi1Dh06RAhZuHAht/D48eMaFSIjI9kSS0vLsLCwb7755scff3zxxRfZcj8/P/ZvNzc3QkhlZaWDg4M+a4mKioqKiuK+q1Ao9u3b9+uvvwYEBOho/+HDh4VCIfezzs7O3t7ehYWFFRUVrq6u+jRvWNd+5swZHUsghEycOLGqqqq2tlZ3NQqHwyBr70t+fn6/dcYIuiuysrL4bggAwEiiZ3cDAAaHnAXA6NTR0dHY2GhlZSWVSgdUwcnJiRCiUqm4hTKZjP3bwsKCEELnIOh3LYSQxsbGTz755NChQxUVFQ0NDWz5o0eP+m2/xqpZt27d4l439NU846xdh+Dg4MLCwitXrmgkEfpaIw7H0Nfelx07duzYsaPfamNHXFwc300AABhh6OBBADAyzMEJMDpZWlrKZLL29vbm5uYBVaiuriaEODs7G2QthJBnn332/fffX7Vq1c2bN3t6ehiG+fTTTwkhDOeRGQKBQHvJdnZ2IpGoq6tLe4RYSEiIPs3jfe0JCQkikejgwYO9vvvXv/5VKBSWlJQQHI7hXzvuDWHh3hAAgEFAwgKAL8hZAIxaS5cuJYQcO3aMWzhz5sw33niDW+GHH35g3+3o6MjOzhaLxREREQZZS3d3d25urrOzc2JioqOjI/13lPvgEsra2rqzs5P+7eXltWfPHkJIdHS0Wq3Ozc3l1vzoo48mTZqkVqv1aRu/ayeEeHp6vvvuuwUFBV999ZXGWzdu3Ni9e/eyZcuefPJJWoLDMaxrBwAAAIARie+UJQAMBtH7uSFyufzo0aNNTU3l5eWrV692cnK6e/cutwJ9UEVTUxP7oIo9e/awC6GzKra1tbEl69evJ4RcvnxZz7WEhoYSQrZt21ZTU/Po0aNTp05NmjSJEHLy5El2mQsWLJDJZPfu3cvLyxOJRNeuXWMYprq6esqUKR4eHseOHWtoaKirq/viiy+sra25G95v84Z17f0+N4RKTk42Nzdfv379jRs3Ojo6Kioq0tPT5XJ5YGBgS0uLxvHC4Rj02nXQ53wZOzDOAgBgEDAHJwBfcNUCMCLp+T9YbW1tUlKSu7u7ubm5XC5/7rnnbt682VcFmUwWERGRnZ1N39KYs3Djxo0MZwA/ISQyMlKftdTU1CQkJLi5uZmbmzs5Ob388svJycl0CexDH0pKSoKCgiQSiZub286dO9nP1tXVrV271sPDw9zc3NHRMTw8nP3nVs/mDdPaqaCgIN3PDWFdvHjxhRdeoM2QSqWzZ89OS0vr6OjQcbxwOAZ6OHRAzoILOQsAgEFAzgKALwLmPy8rAWBEEAgEmZmZy5Yt47shACMAzheurKysuLg49P4AAAMSGxtLCPnuu+/4bgjAmIP5LAAAAAAAAADAFCFnAQAAADBK3L17d9GiRU1NTbW1tYLfzJw5s729nVuN+65AIPD19eWrwX15+PDhF198ERoaOn78eLFY/MQTT6xYsaK4uFi7ZlFRUWRkpJ2dnVQqnT9/vsZkvaxjx455enqKRKK+1qjPcnTXSU5OpvdeAQCAASFnAQAAADAaFBUV+fr6hoeH29raOjg4MAyjVCppeVJSErcmfTc/P9/e3p5hmIKCAp6a3Ke33nrr9ddfX7x48bVr1+rq6r766quioiIfH5/Dhw9zq124cGHOnDlSqfT69etlZWUeHh7z5s376aefuHVKS0sXLVqUkpJCnx7dK32W02+dVatWpaSkvP322wbaBwAAQAghmM8CYETC/fkA+jPm+WJjYzNjxoycnByTXT7msxitmpqavL29IyMjv/jiC7awoKAgMDDQxsamrq5u//79y5cv537k/PnzUVFRtbW1Rm9s/+Lj483MzHbv3s2WFBcXz5gx44knnrh58yYt6enpmT59en19fWlpqVgsJoR0d3d7e3s/evTo1q1blpaWtNrzzz8/ffr0devWTZ48WaVSaT8gWZ/l6Lmu4uLimTNnHjhwAB306IP5LAD4gnEWAAAAACPetm3bVCrVO++8o1FuZWWVkZEhFAoTEhLY//ZNX3p6OjdhQQhRKBRisbi0tJTNuJ07d+7XX3+NiYmhSQRCiJmZ2fLly8vLy48ePcp+8Msvv0xOTtZxV4g+y9FzXQqFIiYm5s0339TOjAAAwOAgZwEAAAAwsjEMk56e7u/vP3HiRO13IyIiNm3a1NzcHBsbqzGxxQjS2tra1tb21FNPCQQCWnLq1ClCiMZkHPRldnY2W8JmGfqiz3L0XBchZOnSpRUVFT/88IPeWwYAALogZwEAAEAIIXV1dWvXrp0yZYqFhcW4ceMWLlx4+vRp+tbmzZvpVIWBgYG05MSJE7TEwcGBlqSmpgoEgtbW1tzcXPoW/V2XlgsEAldXV6VSGRYWJpVKra2tQ0JC2An8hrJ8AEJIcXFxdXW1QqHoq8K7774bHh5+5cqV119/XfeidJwIhw8fZqftvHPnTlxcnJ2dnb29fVRUVGlpKXchNTU1iYmJkydPtrCwcHR0jI6OLioqGuI20jH5GzduZEtKSkoIIa6urtxqLi4uhJABjSjRZzn6r2vGjBmEkB9//FH/BgAAgA7IWQAAABCVSuXn57d///60tLTa2toLFy5YW1uHhYWlp6cTQjZt2sQwjEQiYesvWLCAYRgfHx+2ZN26dbTO3LlzGYZhGIYODqflCoWioaFhzZo1mzdvVqlU586dq6+vDw0NPXv27BCXT4WGhtrb258/f344dxKYrqtXrxKt/6i5hEJhRkaGm5tbenp6RkZGX9V0nwhLlixhGGbx4sWEkKSkpKSkpPv372dmZp46dYo7U0ZVVZWfn19WVtauXbvq6+vPnDlTX18fEBCQn58/6A2srq5OTk6Oj4/nzhPR0NBACOGeOIQQGxsbQsjDhw/1X7g+y9F/XTSRQY8IAAAMHXIWAAAAJCUlpaysbMeOHVFRUba2tp6envv375fL5YmJiTqeNTAgra2tu3btCggIkEgkvr6++/bt6+zsXLNmjUEW3tPTQxMZBlkajDhVVVWEEJlMpqOOg4NDVlaWubl5QkICHTWgTf8TIT4+nn6Z58+fHxkZqVQq2bk8U1JS7t69u3379meeecbGxsbb2/vAgQMMw/Q7xKMvdXV1CxYsmDdvHnd60b7Qs4C9f2TQ9FlOr3VsbW0FAgE9IgAAMHTIWQAAAJBDhw4RQiIjI9kSS0vLsLCwtrY2Q43xlkgkdNA4NW3atIkTJxYXFxvkfxv2p+yhLwpGIjpLhbm5ue5qs2fPTk1NbW1tjY2NbWtr066g/4ng5+fH/u3m5kYIqayspC8PHz4sFAqjoqLYCs7Ozt7e3oWFhRUVFQPdtNbW1oiIiKlTp2ZkZJiZmXHfsrOzoxU06rNv6Umf5QxoXSKRqNfdCwAAg4CcBQAAjHUdHR2NjY1WVlZSqZRb7uTkRAhRqVQGWYv2PzYTJkwghDx48MAgy4exzMrKihDS1dXVb83ExMS4uLirV6++9tprGm8N6ETgjumwsLAghPT09LAL6enpkclkAo5Lly4RQm7dujWg7VKr1bGxsS4uLl9//bVGwoIQ8uSTTxJCNPIg9+/fJ4R4enrqvxZ9ljOgdanV6n4n/gQAAD0hZwEAAGOdpaWlTCZrb29vbm7mltPB8M7OzvSlUCjs7OzkVqC3uHPpGEleV1ence8GzVbQzMXQl///2LvzuKau9H/g5wIhhBACsosoiFWn1ImIjCIgCghScKMgLjj91tovv25I1bbi0nZc6tjBqjPVKdU6dsEK0pdOcVdcwdCCClYtxYIbqyyyyha4vz/ut3fuBAkBQ24Cn/df5uTk3CfBPLl5cu45MJg5ODgQQurq6tTpvHfv3jFjxuzbt++bb77htqv5RlBNKBRaWFgYGRm1t7fTXUyfPl3dp0QIISQmJqa1tTUlJYVdcXbUqFHsui3MaFevXuU+hLkZEBCg/lHUGUf9Y9XX19M0zfxFAADg2aFmAQAAQObNm0cI4W5P2Nramp6eLhKJgoODmRYHBwfmZ1VGeXn5gwcPlMYxNTVl6w5jxoz54osv2LtaWlqys7PZmz///HNpaalMJmO/2zzj+DCYvfDCC6TLLIDumJmZff/992KxePfu3Up3qfNG6FF4eLhCoWC3xWFs3bp1+PDh3IVje/TRRx/dunXr3//+t1AofGoHPz+/559/PjU1ld3AtaOj4+DBg05OTtzLW3qkzjjqH4t5FzN/EQAAeHaoWQAAAJAtW7a4uLjExcUdPXq0oaGhoKBg0aJFZWVlO3fuZCbGE0KCgoJKS0s/++yzxsbGwsLC5cuXs1MkWBMmTCgoKHj48KFcLi8qKvL19WXvkkqla9askcvlTU1NOTk50dHRxsbGO3fuZDs8y/jYN2SQk8lktra2eXl5avZ3c3NLTEzs2q7OG6FHW7ZscXV1Xbp06YkTJ+rq6mpqahITEzds2JCQkMBOl4iOjqYo6u7du90Nsn///r/85S8//vijRCLhXmPC3VTVwMDgyy+/rKmpeeWVV8rLy6urq9988807d+7s2bOHuVhGTeqMo/6xmF1dg4KC1A8AAABU6TptDwB0HyEkOTmZ7ygA9IOa75eqqqq4uDgXFxeBQCCVSoODg9PT07kdamtrly1b5uDgIBKJfHx8srOz2b1I33//faZPfn6+r6+vWCx2cnLatWsX+1iZTObo6Hj79u3g4GCJRCISifz8/DIyMjQ1vq+vr6Wl5ZUrV3p8msnJyfj0H5DWrFljZGRUUlLC3KysrOSe73l4eHR9yOuvv25lZaXUqOKNoLRZ6dq1a+n/vtwpNDSU6VldXb1ixYqRI0cKBAIbG5ugoKAzZ85wj+Lv729mZqZQKLp7OiomSsjlcm7Pa9euhYSEmJubm5mZ+fv7K72taJpOS0vrOsiePXuUuvU4jpp9mAU42trauntqoKciIiIiIiL4jgJgMKJo7IsGoIcoikpOTuZuUw8A3dGF98v48eOrqqr6sGmCxqWkpERFReHTf+Cpq6tzc3MLCwtTZ0NQftXW1g4dOnTx4sV79uzhOxYNy8vLc3d3P3DgwIIFC/iOBTQsMjKSEHLo0CG+AwEYdHBtCAAAAIDek0qlaWlpqampu3bt4jsWVWiajo2NNTc337hxI9+xaFhRUVF4eHh8fDwKFgAAGoSaBQAAAMBA4O7unpOTc+LEifr6er5j6VZFRUVRUVF6erqaG5HokcTExM2bN2/evJnvQAAABhTULAAAAPpRQkICRVF5eXklJSUURa1bt47viGAgc3Z2Pnr0qLm5Od+BdMve3j4jI8PNzY3vQDRv69atmGEBAKBxRnwHAAAAMJCtWrVq1apVfEcBAAAAoJcwzwIAAAAAAAAAdBFqFgAAAAAAAACgi1CzAAAAAAAAAABdhJoFAAAAAAAAAOgiiqZpvmMAgF6jKGry5MnDhg3jOxAAPZCamor3C6u4uDgrKysiIoLvQAAA9ElWVtbkyZMPHTrEdyAAgw5qFgB6KTIyku8QAAB6rby8/Pr16yEhIXwHAgDQa15eXitWrOA7CoBBBzULAAAA0JKUlJSoqCicewAAAICasJ4FAAAAAAAAAOgi1CwAAAAAAAAAQBehZgEAAAAAAAAAugg1CwAAAAAAAADQRahZAAAAAAAAAIAuQs0CAAAAAAAAAHQRahYAAAAAAAAAoItQswAAAAAAAAAAXYSaBQAAAAAAAADoItQsAAAAAAAAAEAXoWYBAAAAAAAAALoINQsAAAAAAAAA0EWoWQAAAAAAAACALkLNAgAAAAAAAAB0EWoWAAAAAAAAAKCLULMAAAAAAAAAAF2EmgUAAAAAAAAA6CLULAAAAAAAAABAF6FmAQAAAAAAAAC6CDULAAAAAAAAANBFqFkAAAAAAAAAgC5CzQIAAAAAAAAAdBFqFgAAAAAAAACgi1CzAAAAAAAAAABdhJoFAAAAAAAAAOgi1CwAAAAAAAAAQBehZgEAAAAAAAAAugg1CwAAAAAAAADQRahZAAAAAAAAAIAuQs0CAAAAAAAAAHQRahYAAAAAAAAAoItQswAAAAAAAAAAXWTEdwAAAAAwYLW3tzc2NrI3m5qaCCGPHz9mWyiKsrCw4CEyAAAA0AcUTdN8xwAAAAADU0VFhaOjY0dHR3cdpk+ffu7cOW2GBAAAAHoE14YAAABAf7Gzs5s6daqBwdPPNyiKWrhwoZZDAgAAAD2CmgUAAAD0oyVLlnR3l6GhYXh4uDaDAQAAAP2CmgUAAAD0o5deesnI6CnrZxkaGs6cOdPKykr7IQEAAIC+QM0CAAAA+pG5uXlISEjXsgVN09HR0byEBAAAAPoCNQsAAADoX9HR0V2X4TQ2Ng4LC+MlHgAAANAXqFkAAABA/woLCzM1NeW2CASCefPmicVivkICAAAAvYCaBQAAAPQvExOT8PBwgUDAtrS3ty9evJjHkAAAAEAvoGYBAAAA/W7RokXt7e3sTXNz8xkzZvAYDwAAAOgF1CwAAACg3wUGBg4ZMoT5t0AgWLhwobGxMb8hAQAAgO5DzQIAAAD6nZGR0cKFC5nLQ9rb2xctWsR3RAAAAKAHKJqm+Y4BAAAABr7MzEwfHx9CiJ2dXWlpqYEBfjgBAACAHuB0AQAAALRhypQpjo6OhJA///nPKFgAAACAOoz4DgAAALShuLj4ypUrfEcBg52np2dJSYmVlVVKSgrfscBgN3/+fL5DAACAnuHaEACAQSElJSUqKorvKAAAdAXOgQEA9ALmWQAADCI4RwcVIiMjCSGHDh3q16OkpqZGRET06yE0haKo5ORk/Bo/8KCGCwCgR3A1KQAAAGiPvhQsAAAAQBegZgEAAAAAAAAAugg1CwAAAAAAAADQRahZAAAAAAAAAIAuQs0CAAAAAAAAAHQRahYAAAAAz+r+/fuzZ8+ur6+vqqqifufu7t7S0sLtxr2XoqiJEyfyFXB3Hj9+/Pnnn/v7+w8ZMkQkEj333HOLFy/Oy8vr2jM3Nzc0NNTCwkIikQQGBmZmZj51wOPHj48ePdrIqNu96tQZR3Wf1atXJycn9/KJAgCAfkDNAgAAAJ5JY2Pjc889FxYWxncgvMnNzZ04cWJQUJC5ubm1tTVN09nZ2Ux7XFwctydzr1wut7Kyomk6JyeHp5C79e6777799ttz5sy5fft2dXX1vn37cnNzPTw8jhw5wu32448/TpkyRSKR/PLLL3fv3h05cuS0adNOnz7N7VNYWDh79uz4+PiKioruDqfOOD32ee211+Lj49evX6+h1wAAAHQIahYAAADwTGia7uzs7Ozs5CsAMzMzHx8fvo5eX18/a9asl1566a233uK2C4VCKyurxMTE7777jq/Y+mbp0qXLly+3t7c3NTX19fU9cOBAR0fHe++9x3bo7Ox89dVXLSws/vWvfzk4OFhbW//zn/90dXVdtmxZa2sr2239+vVTpky5evWqRCJ56oHUGUedPq6urocPH968eXNKSkq/vSoAAMAP1CwAAADgmUgkksLCwuPHj/MdCD8++eST8vLyDz74QKndxMQkKSnJwMAgJiamoKCAl9j6YO/evYmJidwWmUwmEokKCwtpmmZaLl26dOvWrYiICJFIxLQYGhouXLjw4cOHR48eZR/45Zdfrl69WsVVIeqMo+axZDJZRETEypUrFQrFs74EAACgS1CzAAAAAOgjmqb37t07adKkoUOHdr03ODh43bp1DQ0NkZGRSgtb6JGmpqbm5uYXXniBoiim5dy5c4QQpcU4mJvp6elsC1tl6I4646h5LELIvHnziouLjx07pvYzAwAAPYCaBQAAAPTdkSNH2BUlma/l3JZ79+5FRUVZWFhYWVmFhYUVFhYyj0pISGA6DBs2LDs7OyAgQCKRmJqaTp8+nV1ecdOmTUwf9rqPkydPMi3W1tbccZqamjIzM5m7VPyq3x/y8vIqKipkMll3HT788MOgoKAbN268/fbbqoeqrq5esWKFq6ursbGxpaVlSEjI+fPnmbvUeUkZlZWVsbGxzs7OxsbGNjY24eHhubm5z/gcDx06RAhZu3Yt25Kfn08IGTZsGLebo6MjIaRXM0rUGUf9Y40fP54QcurUKfUDAAAA3YeaBQAAAPTd3LlzaZqeM2fOU1vi4uLi4uJKSkqSk5PPnTu3cOFCps+qVatompbJZLW1tcuXL9+0aVN5efmlS5dqamr8/f0vXrxICFm3bh1N02KxmB155syZNE17eHiwLcw4YrHY29ubpmmaprmXBvj7+1tZWWVlZfXf07958ybp8o2ay8DAICkpycnJae/evUlJSd11Ky8v9/T0PHDgwM6dO6uqqn788UdTU9OAgIC9e/cS9V5SQkhZWZmnp2dKSsru3btramouXLhQU1Pj5eUll8v7/AQrKipWr169bNmy+fPns421tbWEEO6fhhBiZmZGCHn8+LH6g6szjvrHYgoZzF8EAAAGDNQsAAAAoL8sW7bMy8tLLBYHBgaGhoZmZ2dXVVVxOzQ1Ne3evZvpM3HixG+//batrW358uUaOXpnZydTyNDIaE9VVlZGCJFKpSr6WFtbp6SkCASCmJgYZtZAV/Hx8Xfv3t2xY0dYWJi5ufno0aMPHDjg4OAQGxurtOmGipc0Pj7+/v37n3766YsvvmhmZubm5nbw4EGapnuc4tGd6urqmTNnTps27fPPP++xM/M6s9eP9Jk64zy1j7m5OUVRzF8EAAAGDNQsAAAAoL94enqy/3ZyciKElJaWcjuIxWJmSj9j3LhxQ4cOzcvL08g3T3aiwbMP1R3mchiBQKC62+TJkxMSEpqamiIjI5ubm7t2OHz4MCEkNDSUbREKhQEBAc3NzUoXO6h4SY8cOWJgYMDddNbe3t7Nze3q1avFxcW9fWpNTU3BwcHPP/98UlKSoaEh9y4LCwumg1J/9i41qTNOr45lZGT01JcXAAD0F2oWAAAA0F+4ExCMjY0JIUpbonb92mlra0sIefToUf9HpwEmJiaEkPb29h57xsbGRkVF3bx5U2lLVEJIa2trXV2diYmJ0p6gdnZ2hJDy8nJuY3cvKTNIZ2enVCqlOK5du0YIuXPnTq+el0KhiIyMdHR0/Oqrr5QKFoSQsWPHEkKU6iAlJSWEkNGjR6t/FHXG6dWxFApFjwt/AgCAfkHNAgAAAHhTXV2tdO0GU61gKheEEAMDg7a2Nm4HZoEDrme/HqHPHBwcCCF1dXXqdN67d++YMWP27dv3zTffcNuFQqFUKm1paWloaOC2M1eF2NvbqzO4UCi0sLAwMjJqb2+nu5g+fbq6T4kQQkhMTExra2tKSgq7pumoUaPYlUGY0a5evcp9CHMzICBA/aOoM476x6qvr6dpmvmLAADAgIGaBQAAAPCmpaUlOzubvfnzzz+XlpbKZDL2m6eDgwPzozqjvLz8wYMHSoOYmpqydY0xY8Z88cUX/Rz1f7zwwgukyyyA7piZmX3//fdisXj37t1Kd82bN48Qwt2ns7W1NT09XSQSBQcHqxlMeHi4QqFgN15hbN26dfjw4dylSXv00Ucf3bp169///rdQKHxqBz8/v+effz41NZXdwLWjo+PgwYNOTk7cy1t6pM446h+L+X/C/EUAAGDAQM0CAAAAeCOVStesWSOXy5uamnJycqKjo42NjXfu3Ml2CAoKKi0t/eyzzxobGwsLC5cvX85OwWBNmDChoKDg4cOHcrm8qKjI19eXadfCviEymczW1jYvL0/N/m5ubomJiV3bt2zZ4uLiEhcXd/To0YaGhoKCgkWLFpWVle3cuZO5QkQdW7ZscXV1Xbp06YkTJ+rq6mpqahITEzds2JAEiRznAAAgAElEQVSQkMBOl4iOjqYo6u7du90Nsn///r/85S8//vijRCLhXmPC3VTVwMDgyy+/rKmpeeWVV8rLy6urq9988807d+7s2bOHuVhGTeqMo/6xmF1dg4KC1A8AAAD0QNfZgwAAMPAkJycj54NqERERERERvX0Us3gka/HixUo7a65du5b+76s/QkNDmcfKZDJHR8fbt28HBwdLJBKRSOTn55eRkcEdv7a2dtmyZQ4ODiKRyMfHJzs7m93r9P3332f65Ofn+/r6isViJyenXbt2sY/19fW1tLS8cuVK314QQkhycnKP3dasWWNkZFRSUsLcrKys5D5ZDw+Prg95/fXXrayslBqrqqri4uJcXFwEAoFUKg0ODk5PT2fuUv8lra6uXrFixciRIwUCgY2NTVBQ0JkzZ7hH8ff3NzMzUygU3T0dFRMl5HI5t+e1a9dCQkLMzc3NzMz8/f2V/nA0TaelpXUdZM+ePUrdehxHzT7MAhxtbW3dPTUW8iEAgB6h6P7cAAwAAHRESkpKVFQUcj6oEBkZSQg5dOiQ1o44fvz4qqqqPmxpoR0URSUnJ8+fP191t7q6Ojc3t7CwMHU2BOVXbW3t0KFDFy9evGfPHr5j0bC8vDx3d/cDBw4sWLCgx87IhwAAegTXhgAAQLcOHjzITAvv1Xxv3XH8+PHRo0ezs+L73EcFMzMz7vx5AwMDS0tLmUz2xhtvKK0aCAOVVCpNS0tLTU3dtWsX37GoQtN0bGysubn5xo0b+Y5Fw4qKisLDw+Pj49UpWAAAgH5BzQIAALq1YMECmqZ7tRGAjigsLJw9e3Z8fDyz80Kf+/SosbHx+vXrhJA5c+bQNN3e3p6fn79hw4b8/PyJEye+8sorT5486fPgoC/c3d1zcnJOnDhRX1/PdyzdqqioKCoqSk9PV3MjEj2SmJi4efPmzZs38x0IAABoHmoWAAAwAK1fv37KlClXr16VSCTP0qe3DA0N7ezs5syZc+7cuffee2///v0LFy7EFPSuEhISKIrKy8srKSmhKGrdunV8R/SsnJ2djx49am5uzncg3bK3t8/IyHBzc+M7EM3bunUrZlgAAAxUfZwKCwAAoMu+/PJLkUj07H2exV//+teLFy/+8MMPBw8eXLhwYf8dSB+tWrVq1apVfEcBAAAAug7zLAAAYABSpxjRrwULQghFUW+99RYhZPfu3f16IAAAAICBCjULAAD4L/n5+XPnzpVKpWKx2NfXNyMjo2ufysrK2NhYZ2dnY2NjGxub8PDw3Nxc5q4jR46wC1Leu3cvKirKwsLCysoqLCyssLCQHaG1tfWDDz4YO3asqanpkCFDZs2a9cMPP3R0dKhzCD3i4+NDCMnKympvb2da8NIBAAAAqA81CwAA+I/ffvvNy8srJycnNTW1oqJi9+7dGzdu5H5hJoSUlZV5enqmpKTs3r27pqbmwoULNTU1Xl5ecrmcEDJ37lyapufMmUMIiYuLi4uLKykpSU5OPnfuHPf6iLfeeuvvf//7P/7xj+rq6l9++WXs2LFz5sy5fPmyOofQJn9/fysrq6ysrL49nFnsUKFQVFVVkUH20gEAAABoAA0AAINAcnKyOjk/MjKSEJKamsq2lJSUCIVCoVDItrz88suEkKSkJLalrKxMKBR6eHiwLcwX77S0NLYlIiKCEFJZWcncdHFxmTJlCvfQo0ePPn/+vPqHUJOjo6OhoWGf+/j5+VlaWl65ckXFw7n7hihhNw0pLS2ldf6li4iIiIiI6LHb4EEISU5O5jsK0Dw18yEAAOgCrMEJAAD/cfLkSUJIcHAw2zJ06NDRo0cXFBSwLUeOHDEwMAgLC2Nb7O3t3dzcrl69WlxcPGzYMLbd09OT/beTkxMhpLS01NramhAyc+bMf/7zn//7v/+7dOlST09PQ0PDX3/9tQ+H6G8XLlx4loeXlZURQgQCAfOsdf+ly8rKYupWwNi+ffuhQ4f4jgI0rLi4mO8QAABAXbg2BAAA/k9ra2tDQ4OJiYmZmRm33dbWltunrq6us7NTKpVSHNeuXSOE3Llzh/tAqVTK/tvY2JgQ0tnZydzctWvX119/XVRUFBAQYG5uPnPmzMOHD/fhEDqOWQ3Ey8tLIBDgpQMAAADoLcyzAACA/yMUCiUSSUNDQ2NjI7dsUVNTw+1jYWHR2NjY3NxsZNT3DxGKopYsWbJkyZL29vYLFy4kJCSEh4dv27ZtxYoVmjoE7zo7O3ft2kUIefPNN4mevHSTJ0/GtAIWRVHvvPPO/Pnz+Q4ENCwlJSUqKorvKAAAQC2YZwEAAP8REhJCfr9ChFFVVcW99IAQEh4erlAoMjMzuY1bt24dPny4QqFQ80AWFhb5+fmEEIFAMGPGDGbLjGPHjmnwELyLj4//6aef5s2bx15tgZcOAAAAoFdQswAAgP/4+OOPhwwZEhcXd+bMmcbGxtu3b0dHRytdKrJlyxZXV9elS5eeOHGirq6upqYmMTFxw4YNCQkJvfpt///9v/9348aN1tbWR48effLJJzRN+/v7a/YQz663+4Z0dnY+evTo3//+d0BAwCeffLJ06dKkpCSKoph7B9VLBwAAAKABfC8CCgAA2qD+Ovm//vrr3Llzzc3NRSKRp6fn0aNHAwICmI+MV199lelTXV29YsWKkSNHCgQCGxuboKCgM2fOMHcp7am5du1amqa5LaGhoTRN5+bmxsTE/OEPfzA1NR0yZMjkyZP37NnT2dnJhqHiEOpIS0vr+pG3Z8+e3vbx9fVVvW+IWCzmPpyiKKlUOm7cuNdff/3q1atd++vyS4d9Q5QQ7BsyQGHfEAAAPULR/30+BAAAAxJz/TZyPqjAXMOC9SxYFEUlJydjPYuBB/kQAECP4NoQAAAAAH7cv39/9uzZ9fX1VVVV7D4v7u7uLS0t3G7ceymKmjhxIl8Bq9be3r59+3YPDw+JRGJraxsSEpKWltZdaWD27NkURW3atInbuHr1amYSBAAAAAM1CwAAAAAe5ObmTpw4MSgoyNzc3Nramqbp7Oxspj0uLo7bk7lXLpdbWVnRNJ2Tk8NTyKo0NTX5+/vv379/+/btjx49ysnJMTMzmz179q1bt7p2/vrrr596cdZrr70WHx+/fv36/o8XAAD0A2oWAACgZ6juffTRR3xHB+oyMzPz8fHR3/GfUX19/axZs1566aW33nqL2y4UCq2srBITE7/77ju+Yuubd99998aNG6dPn546dapIJBo+fPj+/fuFQmHXnqWlpXFxcUuWLOl6l6ur6+HDhzdv3pySktL/IQMAgB5AzQIAAPSMilWaULMAffHJJ5+Ul5d/8MEHSu0mJiZJSUkGBgYxMTEFBQW8xNYHFRUVX3zxxeLFi+3s7NhGsVjc0tLywgsvKHV+7bXXIiMjg4KCnjqUTCaLiIhYuXIlducFAACCmgUAAACAltE0vXfv3kmTJg0dOrTrvcHBwevWrWtoaIiMjFRa2EJn/fDDDx0dHepMbNm3b9+tW7cSEhJU9Jk3b15xcfGxY8c0FyAAAOgr1CwAAACgd5jtVF1dXY2NjS0tLUNCQs6fP8/ctWnTJuY6Hfbr68mTJ5kWa2trpiUhIYGiqKampszMTOYuIyMjtp2iqGHDhmVnZwcEBEgkElNT0+nTp2dmZj77+LojLy+voqJCJpN11+HDDz8MCgq6cePG22+/rXooFX+LI0eOsJdN3bt3LyoqysLCwsrKKiwsrLCwkDtIZWVlbGyss7OzsbGxjY1NeHh4bm5ur57RtWvXCCGWlpYrV650cnIyNjYeMWJEbGxsTU0Nt1txcfHKlSv37dsnkUhUjDZ+/HhCyKlTp3oVAwAADEioWQAAAEAvlJeXe3p6HjhwYOfOnVVVVT/++KOpqWlAQMDevXsJIevWraNpWiwWs/1nzpxJ07SHhwfbsmrVKqaPt7c3c1EPcxUA0y6TyWpra5cvX75p06by8vJLly7V1NT4+/tfvHjxGcdn+Pv7W1lZZWVl9eeL1IObN28SQoYNG9ZdBwMDg6SkJCcnp7179yYlJXXXTfXfYu7cuTRNz5kzhxASFxcXFxdXUlKSnJx87ty5hQsXsoOUlZV5enqmpKTs3r27pqbmwoULNTU1Xl5ecrlc/WdUVlZGCFm6dGlFRcXFixcfPXq0cePGffv2eXl51dXVsd2WLVu2aNEif39/1aM5OjqS318lAAAY5FCzAAAAgF6Ij4+/e/fujh07wsLCzM3NR48efeDAAQcHh9jY2IqKCo0coqmpaffu3V5eXmKxeOLEid9++21bW9vy5cs1MnhnZydTyNDIaH3DfMOXSqUq+lhbW6ekpAgEgpiYmPz8/Kf2Uf9vsWzZMub1DAwMDA0Nzc7OrqqqYge5f//+p59++uKLL5qZmbm5uR08eJCm6R6neHAx17CIRKL9+/ePHDnSwsLiz3/+c3x8fEFBwbZt25g+e/bsuXPnzieffNLjaObm5hRFMa8SAAAMcqhZAAAAQC8cPnyYEBIaGsq2CIXCgICA5uZmTU3mF4vFzNUBjHHjxg0dOjQvL08jX2LZeQTPPlSfMd/wBQKB6m6TJ09OSEhoamqKjIxsbm7u2kH9v4Wnpyf7bycnJ0JIaWkpc/PIkSMGBgZhYWFsB3t7ezc3t6tXrxYXF6v5jJiZL4GBgdzLcGbNmkV+v8TjwYMH77777r59+7hzZFQwMjJ66lMGAIDBBjULAAAAUFdra2tdXZ2JiYnSegTMbhHl5eUaOYqFhYVSi62tLSHk0aNHGhmfdyYmJoSQ9vb2HnvGxsZGRUXdvHlTaUtU0su/BXdOh7GxMSGks7OTHaSzs1MqlXK3DWbWp7hz546az8jZ2ZkQYmVlxW1k/mqVlZWEkLS0tLq6umnTprGHYPY6Xb9+PXPzt99+4z5WoVCIRCI1jw4AAAMYahYAAACgLqFQKJVKW1paGhoauO3MlQj29vbMTQMDg7a2Nm6H2tpapaEoiuruKNXV1UrXbjDVCuY78LOPzzsHBwdCCHehBxX27t07ZsyYffv2ffPNN9x2Nf8WqgmFQgsLCyMjo/b29q6bB0+fPl3NZ8Qsiao0EYb5qzE1lDfffFNpcObpbNy4kbk5atQo9oH19fU0TTOvEgAADHKoWQAAAEAvzJs3jxDC3YeytbU1PT1dJBIFBwczLQ4ODiUlJWyH8vLyBw8eKI1jamrK1h3GjBnzxRdfsHe1tLRkZ2ezN3/++efS0lKZTMZ+iX3G8Xn3wgsvEELUvPLCzMzs+++/F4vFu3fvVrpLnb9Fj8LDwxUKBbszC2Pr1q3Dhw/nrl2q2osvvujo6Hjy5Enu5qxpaWmEkLlz56o5CIv54zKvEgAADHKoWQAAAEAvbNmyxcXFJS4u7ujRow0NDQUFBYsWLSorK9u5cyfzizohJCgoqLS09LPPPmtsbCwsLFy+fDk7RYI1YcKEgoKChw8fyuXyoqIiX19f9i6pVLpmzRq5XN7U1JSTkxMdHW1sbLxz5062w7OMrwv7hshkMltb27y8PDX7u7m5JSYmdm1X52/Roy1btri6ui5duvTEiRN1dXU1NTWJiYkbNmxISEhgF6eIjo6mKOru3bvdDSIUCvfu3VtdXb1gwYI7d+7U1tZ+8803W7ZsmTRpUmxsrJqRsJidVoOCgnr7QAAAGIC6zgMEAICBJzk5GTkfVIuIiIiIiFCnZ1VVVVxcnIuLi0AgkEqlwcHB6enp3A61tbXLli1zcHAQiUQ+Pj7Z2dnsXqTvv/8+0yc/P9/X11csFjs5Oe3atYt9rEwmc3R0vH37dnBwsEQiEYlEfn5+GRkZmhrf19fX0tLyypUrPT5NQkhycrI6L0gfrFmzxsjIqKSkhLnJLPrA8vDw6PqQ119/3crKSqlRxd9CabPStWvX0v99xU1oaCjTs7q6esWKFSNHjhQIBDY2NkFBQWfOnOEexd/f38zMTKFQqH5SV65cCQ4OlkqlxsbGY8eO/eijj548edK1W0xMjNLpaHBwMLdDZGSko6NjW1ub6sP1GfIhAIAeoWhe9/oCAADtSElJiYqKQs4HFSIjIwkhhw4d4jeM8ePHV1VVqb9jRf+hKCo5OXn+/Pn9MXhdXZ2bm1tYWNjnn3/eH+NrUG1t7dChQxcvXrxnzx4tHC4vL8/d3f3AgQMLFizop0MgHwIA6BFcGwIAAACgbVKpNC0tLTU1ddeuXXzHogpN07Gxsebm5hs3btTC4YqKisLDw+Pj4/uvYAEAAPoFNQsAAAAAHri7u+fk5Jw4caK+vp7vWLpVUVFRVFSUnp6u5kYkzygxMXHz5s2bN2/WwrEAAEAvoGYBAAAAOiEhIYGiqLy8vJKSEoqi1q1bx3dE/c7Z2fno0aPm5uZ8B9Ite3v7jIwMNzc37Rxu69atmGEBAABcRnwHAAAAAEAIIatWrVq1ahXfUQAAAIAOwTwLAAAAAAAAANBFqFkAAAAAAAAAgC5CzQIAAAAAAAAAdBFqFgAAAAAAAACgi1CzAAAAAAAAAABdRNE0zXcMAADQ71JSUqKioviOAgBAV+AcGABAL2CvUwCAQWHKlCnJycl8R6Fb2trafv3115s3b968ebOoqIimaWdn53HjxoWEhAwZMoTv6AYmuVy+Y8cO/Ffsb1lZWVlZWbdv366rqzMzM3v++efd3Nzc3NycnJz4Dg0AAKB3MM8CAAAGkY6Ojtzc3LNnz549ezYjI6OlpWXkyJGBgYGBgYH+/v5WVlZ8BzjAMfN9cO6hNUVFRcz/9vT09JqaGltbWz8/v8DAQG9vbzc3N76jAwAA6BlqFgAAMPCx39zOnj37+PFjOzu7qVOnBgYGBgcHjxgxgu/oBhHULPjCVOsyMjIyMzNPnTpVX1/v4ODg4+ODdwEAAOg41CwAAGBgKisry8jIOHv27IkTJx4+fGhmZjZ58mRmSsWECRMoiuI7wMEINQtdoFAo8vLyus428vb2DggIcHR05DtAAACA/0DNAgAABo6qqqrz588zPyZfvXpVJBJ5e3t7e3v7+Pj4+fkJBAK+AxzsULPQNc3NzZmZmcxb5tKlS21tbezVUoGBgZaWlnwHCAAAgx1qFgAAoN+ePHly5coV5kfj69evUxQ1fvx45huXj4+PiYkJ3wHCf6BmocuamprkcnnXt5K3t7efn5+5uTnfAQIAwGCEmgUAAOgf7uT2y5cvt7a24sdhvYCahb6oqqqSy+WZmZlnz569du2aoaGhTCZj3l++vr5CoZDvAAEAYLBAzQIAAPQGu5Tm6dOn6+rq2EUEX3zxxWHDhvEdHfQMNQt9VFFRcenSpbNnz545c+bu3bumpqZTpkzBJVcAAKAdqFkAAIBOY+sU586dq66utrGxmTZtGjZr1FOoWei7oqIiZvGLrkvburu7GxgY8B0gAAAMNKhZAACAzmF/1z19+vS9e/fEYrGXlxe+Fw0AqFkMJGw9MT09vaamhqknMvMvPDw8+I4OAAAGCNQsAABAJzQ2NmZlZTFfgZSun586daqxsTHfAYIGoGYxIHV2dv7yyy/M4hfMdVv29va+vr6BgYFBQUHOzs58BwgAAHoMNQsAAOBNc3Pz1atXma86Fy9e7OjoGDt2LLNERXBwMPYpGHhQsxjwOjo6cnNzmeJjRkZGS0vLyJEjmckXWHcGAAD6ADULAADQqqd+pWHmU/j7+1tZWfEdIPQj1CwGFW5R8tKlS21tbeybPSAgYMiQIXwHCAAAegA1CwAA0Ab20vezZ88+fvzYzs5u6tSpzHyKESNG8B0daAlqFoNWU1OTXC5nMsD169cpiho/fjwz/yIoKEgqlfIdIAAA6CjULAAAoL+UlZVlZGScPXv2+PHjxcXF3C0GJkyYQFEU3wGCtqFmAYSQhoaGH3/88amL1/j4+JiYmPAdIAAA6BDULAAAQJMqKysvXLjAXPdx+/ZtkUjk7e3N/Jrq5+cnEAj4DhD4hJoFKGE3CWIzxoQJE5hFbbD4LgAAENQsAADg2T111jd+NYWuULMAFUpLS5nFL06ePPngwQNscgwAAAQ1CwAA6BuFQpGXl8fUKZRW15sxY4aFhQXfAYIuQs0C1MSugHPu3Lnq6mpra+vJkycz8y9wZRkAwKCCmgUAAKirs7Pzl19+YX4IPXXqVH19vYODA/MtArsYgjpQs4A+YOsXZ86cqa2tZVfwnTFjhouLC9/RAQBA/0LNAgAAeqD0g6eNjc20adOYKRUjR47kOzrQJ6hZwLNgd0rOyMi4ePFiQ0MDWzYNCQlxcnLiO0AAANA81CwAAOAp2IXxTp06df/+fVxYDhqBmgVoCvfytMuXL7e2trKXp/n7+1tZWfEdIAAAaAZqFgAA8H8aGxuzsrKeugEhFvAHjUDNAvrDkydPrly5wsy/+Omnnzo6OsaOHcvMv8DyOgAA+g41CwCAQa25uTkzMzMjIyMzM/PixYsdHR3u7u7M1qTBwcHm5uZ8BwgDCmoW0N+Uaq8GBgbYxggAQK+hZgEAMOiw14QzP0u2tLSwc6oDAgKGDBnCd4AwYKFmAdr06NGjixcvMjXZq1evGhkZYe4YAIDeQc0CAGCw6G7t/eDg4BEjRvAdHQwKqFkAX8rKyjIyMrBGDwCA3kHNAgBgIGNP048fP15cXCyRSCZNmsScpk+YMIGiKL4DhMEFNQvQBUwBNyMj49y5cyUlJUiMAAC6DDULAICBprKy8sKFC8wZ+e3bt0Uikbe3d2BgoLe39+TJk42MjPgOEAYv1CxA17AT0M6ePfv48WNbW1s/Pz8mYbq5ufEdHQAAoGYBADAgNDU1yeVy5rT7+vXrFEVh2TnQQahZgM5iFvphFr84depUfX29g4MDs/nIzJkzhw8fzneAAACDFGoWAAD6SqFQ5OXlMXWKS5cutbW1sUtpYns/0E2oWYBe4GbXy5cvt7a2stl1+vTp1tbWfAcIADCIoGYBAKBPOjs7f/nll8zMTGYlOe4vgaGhoY6OjnwHCKAKahagd548eXLlyhV2Q+j29na2fhEYGGhpacl3gAAAAxxqFgAAeoC94vrcuXPV1dU2NjbTpk1j5lO4uLjwHR2AulCzAL3W2NiYlZXV9So8b2/vadOmSSQSvgMEABiAULMAANBRFRUVly5deurOfFjZHvQUahYwYFRWVmZlZTGz3q5du2ZoaCiTyZgU7evrKxQK+Q4QAGCAQM0CAECHcH/EUzoJnjp1qrGxMd8BAjwT1CxgQCovL798+fLZs2dPnz597949U1PTKVOmMPMvJk2aJBAI+A4QAECPoWYBAMCz5uZm5pe6jIyMn376qaOjw93d3dvb28fHZ+bMmZhsDAMJahYw4BUVFTGLXxw/fry4uNjMzGzy5MmYIgcA0GeoWQAA8IDZVI+ZT5GRkdHS0sIu6hYQEDBkyBC+AwTQjMrKysOHD7M3c3Jy9uzZk5iYyLZIJJKFCxfyERpAv2OXIkpPT6+pqWGWImJK0h4eHnxHBwCgH1CzAADQHvb89cyZM7W1tfb29r6+voGBgTNnzhw+fDjf0QFoXmtrq62tbWNjo6GhISGEOetgf2pub29/+eWX9+/fz2OEAFrA3fLp9OnTdXV1bP4PDg4eMWIE3wECAOgu1CwAAPpXaWkpc57KzBOWSCSTJk3CPGEYPF599dVvv/22ra3tqfeeOnUqKChIyyEB8EihUOTl5XWdZ+ft7R0QEIAtqwEAlKBmAQCgeZWVlRcuXGBOSYuKitj12AIDA93d3Q0MDPgOEEB70tPTAwMDn3qXhYVFZWWlkZGRlkMC0BHNzc1Xr15l6tqXLl1qa2tjrxMMDAy0tLTkO0AAAP6hZgEAoBlNTU1yuZzd8sPAwGD8+PHY9w6gs7PT3t6+srJSqV0gEMTExPzjH//gJSoAXcP9ELl+/TpFUePHj2cWvwgODjY3N+c7QAAAfqBmAQDQd9wpvko/kc2YMcPCwoLvAAF0wjvvvLNr16729nal9szMzClTpvASEoAua2ho+PHHH5+677WPj4+JiQnfAQIAaA9qFgAwqDU3N//lL3/505/+FB4eruZDOjs7r1+/zmxld+rUqfr6+pEjRzI/hYWGhuJSZICufvrpp0mTJik1Dh06tLi4GEu6AKhWUVFx6dIlZvHmu3fvikQib29v5kPHz89PIBCoP9Ts2bM//PBDbFkCAPoFNQsAGLwuX74cHR394MGDmJiYzz//XHVnpS3rbG1t/fz8mPkULi4u2gkYQH85Ozvfv3+fvWlsbLxy5cqPP/6Yx5AA9A67qPOJEycePnxoZmY2efJkNRdLKioqcnV1NTQ0XL9+/Zo1a3pV7AAA4BFqFgAwGDU3N69du3bHjh2GhoYKhWL48OHcb1Os8vLyy5cvnz179uTJkw8ePBCLxV5eXtjyA6AP1q9fv3XrVu7lITdu3Bg3bhyPIQHoNbaSfu7cuerqamtr68mTJ/v4+HT3CfXll1/GxMR0dHQYGhqOHTs2KSlJJpPxEjkAQK+gZgEAg05WVlZ0dPT9+/cVCgXbePfuXWdnZ6LyKuKpU6caGxvzFjeAPsvPz//DH/7A3hw1atSdO3d4jAdgIGHrF2fOnKmtrbW3t/f19VWaCbho0aJDhw4xH3xGRkadnZ3vvvvuhg0b8LkGADoONQsAGERaWlo++uijv/3tbwYGBtyChaGh4fLly42MjNLT05nV2idMmBAQEBAQEODt7S0SiXiMGWDAcHNz++WXX2iaFggEH3300Zo1a/iOCGCgUSgUP/300/nz58+dO3flypWWlpZRo0ZNnz7d39//rbfeqq6u5nY2NDR87rnnkpKSJkyYwFfAAAA9Qs0CAAaLvLy8xYsX5+fnd3R0KN1lZGTU0dHh4uLCzKcICAgYMmQIL0ECDGBbt25dt26dQqGgKKqwsBALwQD0q647W3XtY2RkRNP0qlWrMOECAHQWahYAMPC1t7d/+umna9eupSiKO72Cy9LSsrq6GktUAPSfB1vv/DEAACAASURBVA8eODs70zTt4eGRk5PDdzgAg8i2bdtWr17d3SegoaHh6NGjk5KS3N3dtRwYAECPVC0vDAAwAPz8888eHh5r167t6Ojo7nSNEPL48eNffvlFm4EBDDbDhw9ndjx9+eWX+Y4FYHDJyMhQ8TtlR0fHnTt3PD09V69e/dTpGAAAPMI8C42Ry+Wffvop31EAwH90dnbm5+cz18/32JmiKJlMNmrUKC0EphFeXl4rVqx4xkE+/fRTuVyukXgA1FFYWJibmxsaGmpiYsJ3LDCIrFixwsvL6xkHiYyM1Egw2kfT9A8//MDdtUcFCwuLP/3pT+bm5v0dFQBohEbym47DPAuNefjwYWpqKt9RgPakpqYWFxfzHYWuKC4u1sH//5WVlcw+pvb29hYWFiKRyNDQkNuBoiiD39E0XVFRwVeovZWVlaWRWoNcLs/Kynr2cQBU4OaHYcOG2dnZDfKCRVZWFt532pSamvrw4UONjKOnn/t1dXVMwYKiKOaDT6mDQCAQi8VDhgxxdHS0srIqLS1Vs8ABunn+wyPkNy3TVH7TcUZ8BzDQHDp0iO8QQEsoinrnnXfmz5/PdyA6ISUlJSoqSi/+/7e0tFRVVVVVVVVUVFT9rrKysqKiQqFQ6MVTIBr9uW/y5Mn68qxBTynlh99++02PJjT1B+b9i/ed1mhwoSI9/dz/9ttvGxsbbWxs7O3t7ezsbGxsrK2t7ezsbG1tmX8bGeEbQR/p0fmPdiC/adkgWYgNGQoABhcTE5Nhw4YNGzaM70AABqlBXrAA0L7o6Ojo6Gi+owAA6CNcGwIAAAAAAAAAugg1CwAAAAAAAADQRahZAAAAAAAAAIAuQs0CAHhw//792bNn19fXV1VVUb9zd3dvaWnhduPeS1HUxIkT+QpYtfb29u3bt3t4eEgkEltb25CQkLS0tO42WJ09ezZFUZs2beI2rl69Ojk5WSvBAoCeQcJEwgQYqJDfkN/UgZoFgFY1NjY+99xzYWFhfAfCp9zc3IkTJwYFBZmbm1tbW9M0nZ2dzbTHxcVxezL3yuVyKysrmqZzcnJ4ClmVpqYmf3///fv3b9++/dGjRzk5OWZmZrNnz75161bXzl9//XVaWlrX9tdeey0+Pn79+vX9Hy+A3kDCJEiYSJgwQCG/EeQ35De1oWYBoFU0TXd2dnZ2dvIVgJmZmY+PD19HJ4TU19fPmjXrpZdeeuutt7jtQqHQysoqMTHxu+++4yu2vnn33Xdv3Lhx+vTpqVOnikSi4cOH79+/XygUdu1ZWloaFxe3ZMmSrne5uroePnx48+bNKSkp/R8ygH5AwkTCRMKEgQr5DfkN+U19qFkAaJVEIiksLDx+/DjfgfDmk08+KS8v/+CDD5TaTUxMkpKSDAwMYmJiCgoKeImtDyoqKr744ovFixfb2dmxjWKxuKWl5YUXXlDq/Nprr0VGRgYFBT11KJlMFhERsXLlSoVC0Y8RA+gPJEwkTCRMGKiQ35DfkN/Uh5oFAGgPTdN79+6dNGnS0KFDu94bHBy8bt26hoaGyMhIpesYddYPP/zQ0dGhzi8V+/btu3XrVkJCgoo+8+bNKy4uPnbsmOYCBAB9hYSJhAkwUCG/Ib/1CmoWANpz5MgRdvUgJgVzW+7duxcVFWVhYWFlZRUWFlZYWMg8KiEhgekwbNiw7OzsgIAAiURiamo6ffr0zMxMps+mTZuYPmyuPHnyJNNibW3NHaepqSkzM5O5y8jISMuvQF5eXkVFhUwm667Dhx9+GBQUdOPGjbffflv1UNXV1StWrHB1dTU2Nra0tAwJCTl//jxzlzqvKqOysjI2NtbZ2dnY2NjGxiY8PDw3N7dXz+jatWuEEEtLy5UrVzo5ORkbG48YMSI2Nrampobbrbi4eOXKlfv27ZNIJCpGGz9+PCHk1KlTvYoBYEBCwkTCRMKEgQr5DfkN+a13aNAQZolXvqMA7SGEJCcn9+GBc+bMIYQ0NzcrtcyZM+fKlSuNjY1nzpwRiUSenp7cR8lkMrFY7OXlxfTJzs7+4x//aGxsfOHCBbaPWCz29vbmPsrDw4NZrEhFH8b06dOHDBkil8v78Ixotf//f/PNN4SQjz/+WKk9OztbKpUy/66srHRyciKEfPvtt0wLu+QSq6yszMXFxc7OLi0tra6u7tdffw0PD6coas+ePWyfHl/V0tLSESNG2NnZHTt2rKGh4ebNm35+fiYmJleuXFH/iTNHsbe3X7x4cWFh4ePHj7/66iuxWDx69Oja2lq2W3Bw8BtvvMF9ETZu3Nh1tLq6OkKIr69vj8eNiIiIiIhQP87+HgdAhWf5fByQCVPN9x0SJq2hhNnnz+t+GgcGEuQ3JchveprfdBzmWQDoimXLlnl5eYnF4sDAwNDQ0Ozs7KqqKm6Hpqam3bt3M30mTpz47bfftrW1LV++XCNH7+zsZJKCRkbrTllZGSFEKpWq6GNtbZ2SkiIQCGJiYvLz85/aJz4+/u7duzt27AgLCzM3Nx89evSBAwccHBxiY2MrKiq4PVW8qvHx8ffv3//0009ffPFFMzMzNze3gwcP0jTdY0Wfi/l5RCQS7d+/f+TIkRYWFn/+85/j4+MLCgq2bdvG9NmzZ8+dO3c++eSTHkczNzenKIp5lQBABSRMBhImEiYMPMhvDOQ35DcWahYAusLT05P9N1NXLi0t5XYQi8XMVDHGuHHjhg4dmpeXp5GMduHChZqaGi8vr2cfSgUmoQsEAtXdJk+enJCQ0NTUFBkZ2dzc3LXD4cOHCSGhoaFsi1AoDAgIaG5uVppHp+JVPXLkiIGBAXebMXt7ezc3t6tXrxYXF6v5jMRiMSEkMDCQO69y1qxZ5PcZfQ8ePHj33Xf37dvH9OyRkZHRU58yAHAhYbKQMNU8OoC+QH5jIb+pefQBDzULAF3BLTYbGxsTQpR2wLKwsFB6iK2tLSHk0aNH/R+dZpiYmBBC2tvbe+wZGxsbFRV18+ZNpR2wCCGtra11dXUmJiZKlwIyCzWXl5dzG7t7VZlBOjs7pVIpxcFcjnjnzh01n5GzszMhxMrKitvI/F0qKysJIcxkxWnTprGHYLa2Wr9+PXPzt99+4z5WoVCIRCI1jw4waCFhciFhAgwkyG9cyG9AULMA0CPV1dVKU/WYDycmIRJCDAwM2trauB1qa2uVBqEoqj9j7IGDgwMhhLlIr0d79+4dM2bMvn37mOv9WEKhUCqVtrS0NDQ0cNuZSYD29vbqDC4UCi0sLIyMjNrb27teNTd9+nQ1nxGzxpXSLxvM34X5yHzzzTeVBle6fHHUqFHsA+vr62maZl4lAHgWSJgMJEyAgQf5jYH8NnigZgGgN1paWrKzs9mbP//8c2lpqUwmYzOag4NDSUkJ26G8vPzBgwdKg5iamrIfY2PGjPniiy/6Oer/wuxQreZEOzMzs++//14sFu/evVvprnnz5hFCuFtAtba2pqeni0Si4OBgNYMJDw9XKBTsUtuMrVu3Dh8+XP0NsV988UVHR8eTJ09y9+JKS0sjhMydO1fNQVjMn6/rPt4A0FtImCwkTIABBvmNhfw2SKBmAaA3pFLpmjVr5HJ5U1NTTk5OdHS0sbHxzp072Q5BQUGlpaWfffZZY2NjYWHh8uXL2Yo7a8KECQUFBQ8fPpTL5UVFRb6+vky7v7+/lZVVVlZWvz4FmUxma2ubl5enZn83N7fExMSu7Vu2bHFxcYmLizt69GhDQ0NBQcGiRYvKysp27tzJFLPVsWXLFldX16VLl544caKurq6mpiYxMXHDhg0JCQnstYjR0dEURd29e7e7QYRC4d69e6urqxcsWHDnzp3a2tpvvvlmy5YtkyZNio2NVTMSFrOxVlBQUG8fCABKkDBZSJgAAwzyGwv5bbDoOgcG+gZ7nQ42pPd7CzELBbEWL14sl8u5LWvXrqX/e7JfaGgo81iZTObo6Hj79u3g4GCJRCISifz8/DIyMrjj19bWLlu2zMHBQSQS+fj4ZGdne3h4MOO8//77TJ/8/HxfX1+xWOzk5LRr1y72sb6+vpaWlr3a1YlL/f//a9asMTIyKikpYW4y1/ixPDw8uj7k9ddfV9raiqbpqqqquLg4FxcXgUAglUqDg4PT09OZu9R/VZk9vUeOHCkQCGxsbIKCgs6cOcM9ir+/v5mZmUKhUP2krly5EhwcLJVKjY2Nx44d+9FHHz158qRrt5iYGKUMHBwczO0QGRnp6OjY1tam+nA09joFvdK3z8cBnDDVf98hYWokYRLsdQr9BvlNCfKbnuY3HYfv2BqDmsVgo+UcwXxEae1wvaX+///a2lpHR8eYmJj+DunZPX78WCQSLVu2TDuHy83NpSjqu+++U6czahagR7T/+ajjCVP99x0SZnd6lTBRs4D+g/ymBPnt2fGS33Qcrg0BAK2SSqVpaWmpqam7du3iOxZVaJqOjY01NzffuHGjFg5XVFQUHh4eHx+/YMECLRwOAPQCEuZTIWECDADIb0+F/PZUqFmA9piZmXH3EDIwMLC0tJTJZG+88cbVq1f5jg60x93dPScn58SJE/X19XzH0q2KioqioqL09HQ1151+RomJiZs3b968ebMWjvXsDh48yLyLmb3KoD8gYQIDCbMrPUqYvcqWBw4ckMlkIpGIeUh+fj4hRC6Xh4WFWVtbGxgYMO0TJ07UbJAJCQnMyMOGDdPsyN1BfgMG8ltXepTftAk1Cz3T2Nj43HPPhYWF6eBo6hzu+vXrhJA5c+bQNN3e3p6fn79hw4b8/PyJEye+8sorT5480U4k+oU5mcjLyyspKaEoat26dXxHpAHOzs5Hjx41NzfnO5Bu2dvbZ2RkuLm5aedwW7du1aOC+oIFC2iaDggI4DuQgQwJs2+QMHmBhNmdp2bLp559XbhwYfHixaGhoZWVlXfu3GE2gMjOzp42bVpzc/O5c+eam5sfPXq0fv16jQe5atUqmqZlMpnGR+4O8lvfIL/xAvlNF6BmoWdomu7s7Ozs7Oy/0czMzJgdhvuboaGhnZ3dnDlzzp0799577+3fv3/hwoX0fy+NA+T3kwnWpk2b+I4IQEdpLX2pT1MhIWGqCQkTdN9Tz74OHTpECFm+fLmZmdmoUaNKS0vHjh27ffv2tra2b7/99o9//KNQKLSxsfnggw+YBRS0ACeEugb5DQYt1Cz0jEQiKSwsPH78uA6O9iz++te/Tpo06Ycffjh48CDfsQAA6DQkTAC99tSzr4cPHxJCrKysuI35+fm2trbMnAuGkZGRq6urduLkBfIbAHSFmgXoBIqi3nrrLULI7t27+Y4FAECnIWECDDxPnULb1tYmEom0HwyPkN8AoCvULLSKu8pRdnZ2QECARCIxNTWdPn16ZmYm0+fIkSPsokS//vrr/PnzraysmJt79+5l72ppaWGHZbYUdnV1NTY2trS0DAkJOX/+fB9GY8JramrKzMxk2o2MjGpra7nrJDHz0BQKBdsSERGhkReHmX+YlZXV3t7OtFRWVsbGxjo7OxsbG9vY2ISHh+fm5nZ9Xvfu3YuKirKwsLCysgoLCyssLGTHbG1t/eCDD8aOHWtqajpkyJBZs2b98MMPHR0dbAcVhwAArvz8/Llz50qlUrFY7Ovrm5GR0bWPijeUOtmPEKJQKJKTk2fMmGFvby8SicaNG7dz5072VF5FQlu9enXX9KX0kPv370dFRUkkEisrqyVLljx+/PjevXuzZs2SSCQODg6vvfZaQ0ODmk9HnRT01IyqqT8HEiaAzlKdLbnvR+bsKzU1laKoY8eOEUIEAgH3pOvWrVv379+nulC6KIA9DxQKhcOGDQsMDNy/f39zczMhZNOmTcxD2Ks8Tp48ybRYW1t39xRwQoj8BqBb+mUH1UFJ/f2ZZTKZWCz28vK6cuVKY2Njdnb2H//4R2Nj4wsXLrB95syZQwjx8/M7f/58U1NTVlaWoaFhZWUle1dzczPTs6yszMXFxc7OLi0tra6u7tdffw0PD6coas+ePX0YjaZpsVjs7e2tFHNwcLCBgcFvv/3GbfTy8kpKSmJvTp8+fciQIXK5XMVz5y65pIT5cCWElJaW0jRdWlo6YsQIOzu7Y8eONTQ03Lx508/Pz8TE5MqVK0rPa86cOcwreebMGZFI5OnpyXZYtmyZVCo9ffr0kydPysvLV61aRQg5f/48c686h1CBDI79kNWk/f3JBzn19z/XyDh37tyxsLBwdHQ8ffp0Q0PDjRs3goKCnJ2dhUIh20edN1SP2S8tLY0Q8vHHH9fU1FRWVv797383MDBQuoJXRUJ7avpiHxIeHp6Tk9PY2Pj1118TQkJCQubMmXP9+vWGhobPP/+cEPLOO+/06un0mIJUhDSoEibygxJNvX9BTZr6vFZnHHWyJf20s6/Q0FBCSHt7O7ebm5vbiBEjuC2XL18mhGzcuJFtYc4D7e3t09LS6uvry8vLmR0Zt2/fzvbpmog8PDysrKy4LTKZzNHRkduCE0Lktz5AftOyQfJ9BO8xjelVzYIQcv36dbblxo0bhBCZTMa2MMn3+PHjXR+u9Dn3P//zP4SQ7777ju3Q0tIydOhQkUhUXl7e29Hobj6iTp06RQh544032JaMjAxHR8e2tja2xc/Pz9LSUnV+V/ERxa4RzXxEvfzyy4QQ7kdgWVmZUCj08PBQCj4tLY1tYWr8zLcXmqZdXFymTJnCPcro0aPZjyh1DqHCIMkRasJntpZpuWYRGRlJCElNTWVbSkpKhEIh9yxcnTdUj9kvLS1t2rRp3ENHR0cLBIK6ujq2RUVCU12zOHbsGNvCLAB+8eJFtsXFxWXMmDG9ejo9piAVIQ2qhIn8oATn9FqmzZqFOtmS1mjNgjkPVAps5syZ/VSzwAmhEuQ3JchvWjZIvo/gPaYxvZ1nodQ4dOhQNjvTvyffqqqqrg9X+pyTSqWEkPr6em6fJUuWEEK++uqr3o5Gd3+GPW7cOFNTU3aQOXPm/PWvf1Xn+XKp+IhipvAJBALmY08qlRoYGHC/qNA0PWHCBELIw4cPucGzpRmapt955x1CSF5eHnPz9ddfJ4S89tprcrlcoVAoHVGdQ6hAAHilzZqFRCIhhDQ0NHAbx40bxz0LV+cNpU72U/K3v/2NENL197SnJjTVNYuKigq2ZcaMGYSQpqYmtsXHx0cikfTq6fSYglSEpI4BkzC1ts0BQHe0VrNQJ1vSGq1ZPPU8UIkGaxY0Tgj/G/Ib8G4w1Cw0dm0t9IqFhYVSi62tbWlp6aNHj7irQ4vFYtXjtLa21tXVmZiYMJ+RLDs7O0JIeXk5t7HH0VSLi4t79dVXd+/evX79+oKCgnPnzv3rX/96lgGVMBd8enl5CQQC5nkRQphPYiV37twZNmwYe5Pbx9jYmHAWstq1a5eXl9dXX33FbI3u6+sbExMzb9488vtLp+YhuhMXF+fl5dWLJzlwyeXyHTt24JNba7Zv3661Y7W2tjY0NJiYmJiZmXHbbW1tCwoK2D5qvqFUZ7+6urpt27YdPny4uLi4traW7cP+7MbqQ0Ljbv9uYGBgaGhoamrKthgaGrKpQ1MpqP/oY8JEfmAx71/mOxVoQVRUlHYOpE621PgRn3oe2K9wQtgV8hsL+U3LtJbf+IWaBT+qq6tpmqYoim159OgRIcTW1rZX4wiFQqlUWldX19DQwP24qqioIITY29v3ITZuVFyLFy9es2bNZ5999t57723btu3ll1+2tLTsw/hP1dnZuWvXLkLIm2++SQgRCoUWFhaNjY3Nzc3PsmodRVFLlixZsmRJe3v7hQsXEhISwsPDt23btmLFCo0cwsvLa/78+X0Ob4DZsWMHXg2tOXTokNaOJRQKJRJJQ0NDY2Mj90S8pqaG20fNN5Tq7Ddr1qzLly/v3Llz4cKF1tbWFEXt2LGDWWZCnVC7S1+9pakUpMGQuPQ0YSI/sJj3L14QrdHaOb062VLjR3zqeaASAwODtrY2bgu3KNwdnBCqD29nFvKblg2SmgX2DeFHS0tLdnY2e/Pnn38uLS2VyWTcSRZqYorEzHLTjNbW1vT0dJFIFBwc3IfYTE1N2Q+2MWPGfPHFF8y/hULhG2+88ejRo23btiUlJS1fvrwPg3cnPj7+p59+mjdvHnMhKCEkPDxcoVBwNxQghGzdunX48OEKhULNYS0sLPLz8wkhAoFgxowZzOLS7GulkUMADAYhISGEkJMnT7ItVVVVv/76K7ePmm8oFdmvo6MjMzPT3t4+NjbWxsaGOV1mF2NTR3fpqw80lR80GBILCRNAZ6mTLTWLOQ88fvw4t9Hd3Z37Q7eDg0NJSQl7s7y8/MGDBz2OjBNC5DcAHYGaBT/+f3v3HtPU9cAB/F6lhVqkICogoiIGWZgiVjPdaFQ0bTYUB5NoHCaLwzB1g25iBEVNfBEZWUYCKoOxxfkkLrDAxhZG9A9YTYoLOOe0DnyVh8CQ8phWCPf3x8nvprtAe6FPLt/PX/be09NzL/R7L8dzz5HJZAcPHtRoNP39/XV1dYmJiWKxODc3dxxVZWVlBQcHq9XqioqK3t5enU63ffv21tbW3Nxc8oTIWC1fvlyn0z19+lSj0TQ1NSkUCnbXnj17JBJJZmbmhg0bFi1axHljdHS0r6/vzZs3eX7Q0NBQe3v7Dz/8sH79+uzs7J07d168eJHt1M/KygoJCdm5c2dlZaXBYOjq6iooKDh27FhOTs6YesE/+uij27dvG43G9vb27OxshmGio6Nt+xEAgnfq1KkZM2ao1eqqqqq+vr67d+8mJiZyBj/z/EKZSb+pU6euXbu2ra3t888/7+zsfPHixfXr18mKHjyZia+xslU+jNYkBCaAIPFJS9si94GffvopWfNCr9fv2bOntbXVtM9CqVS2tLTk5eX19fU1NjampqbyGdiLG0LkG4CrcOZkGsIypjk4AwMD7969q1Kppk+fLpFI1qxZU1NTQ/ZqNJrRfkalpaWm299//32yvbOzU61WBwcHi0QimUymUqmqq6vHXdu9e/cUCoVUKg0KCsrPz+c0fteuXdR/J9tnKRQK89NEc54/p2laJpMtWbJk9+7dt27dGl6erDe+cOFCkUg0a9YspVJZVVU14nEdOnSI+e/Q8ZiYGIZh6uvrk5OTX3vtNbIc96pVqwoLC4eGhvh8hEXU5JjzhifMm+1gDl43hGGY+/fvv/vuu15eXmQBuYqKCvJUMEVRH374ISlj8QtlPv0Yhuno6EhOTg4KChKJRH5+fh988EF6ejr5FLlcbibQiOHxNTwrTEd5UBSVlZVF5rRjHT161OLh8IygEZtETKrARD5wYF59B7PV9ZpnPebTcvjd1/Bn/bRaLZl+mJNL8+fPN93Y2tpKPtH0PjAgIGDbtm06nc60Sd3d3UlJSQEBARKJJCoqSqvVyuVyUsmBAwc4n0UigsENIfJtXJBvDmarfHNxNIPlD2ykpKRk69atfM7nsmXLOjs79Xq9A1plc998801+fn5dXZ2zG+J8NE1fvXoVD+wR/H//wSbIoFnrZ7WwVT08Tej0g3FDPnA4+HsHtrpe47pvCjeEBPKNA/nmYJMkl/BsCIzNuXPnPvvsM2e3AgTo8ePHsbGxPT09nZ2d9P9FRka+fPnStJjpXpqmV6xY4awGW/TTTz+FhoaOOKw0PT0dc4wDwLgJLzCJ2NhYmqZPnDhhuhGB6ZpwQwh2Iph8YximtrZ27969oaGh7u7us2fPjoqKunDhgmkPF/KNJ/RZgGVFRUVxcXF9fX3nzp17/vy54HvywPHq6+tXrFihVCq9vLxmzpzJMAwZwF9fX69Wq01Lkr0ajYYsLO+a/8PT2NgYGxubkZFBVvAZbteuXRkZGYcPH3ZwwwBAAAQWmKzz58+Xl5cP347AdB24IQR7E1K+3b9/PyoqSqfTXbt2zWAw3Lx5c968eTt27Ni/fz9bBvnGE/osHConJ4em6YaGhubmZpqmMzMznd0ivsrKynx8fM6ePXvlyhVMR+R4np6eUVFRE7d+83p6ejZt2vTee+99/PHHptvd3d19fX0LCgouX77srLaNz+HDh998881bt26NtvJcSEhIaWnpyZMnS0pKHNw2Z5m46QcTDgLTWW2zRktLi1qt3rFjx/BdkzAwXRluCJ0L+easto2Pm5tbSUnJ0qVLPTw8Fi5c+O233/r6+ubl5RmNRlIA+cYT+iwcKi0tzXQ2Ec7oR5eVlJTEMMzAwEBDQ8Py5cud3RwQmuzs7La2tiNHjnC2e3h4XLx4ccqUKcnJyTqdziltG5+vv/46PT3d/M1cRETEli1b9u3bN0nWUZug6QfgaoQXmMSuXbsSEhKUSuWIeydbYLos3BCCXQks38LCwgYGBnx8fNgtYrE4KCjIaDSaPueCfOMDfRYA4EwMwxQVFb3xxhtz5swZvlelUmVmZvb29iYkJHCeY3RlEomET7G4uDi9Xs+uDw8AYJ4gA5OiqOLi4j///DMnJ8dMGQQmgLAJNd9MdXd3P3jwIDIyUiaTmW5HvlmEPgsA+yKrZ4WEhIjFYh8fn7fffvv69etk14kTJ8i8QewwvJ9//plsmTlzJtlCRtT39/fX1taSXeR/78l2mqbnzp2r1WrXr18/ffr0adOmrVu3rra21vr6HaahoeHZs2cRERGjFTh69KhSqbx9+/Ynn3xiviozp7qsrIydpenRo0dbt2719vb29fXduHFjY2OjaSUdHR0pKSkLFiwQi8WzZs2Kj4+vr6+3/jBHtGzZMoqifvnlFzvVDzDhIDDNE2Rg6vX6ffv2FRcXj/YwHYHAhIkO+WaeIPON1dPTU1tbGxsb6+/vf/78ec5e5JtldltFddLB+syTDcVjPeTW1tbg4GA/P7/y8nKDwXD//v34+HiapgsLC9kyUqn0rbfeMn2XXC4n8wmZKUNERERIpdLVq1f/9ttvfX19Wq126dKlYrH4xo0bNql/3bp1M2bM0Gg05g+TseL3scu7oQAACHBJREFU/7vvvqMo6tSpU5ztWq1WJpORf3d0dAQFBVEURSZbZkymXGLxOdWbN2+mKGrz5s3kdFVVVUkkkpUrV7IFWlpa5s+f7+fn9+OPP/b29t65c2fNmjUeHh5mFpk3LzAwcOrUqaPtNRgMFEUpFIpx1Gyr9c+xjjo4AM98mDyBOe7vnSADU6VS7dmzx/QAjx8/PryYNYHJ53rtyHpASJBvHMi34Y4fP07+7l67du3t27eHF3CFfHNxGGcBYEcZGRkPHz788ssvN27c6OXlFRoaeunSpYCAgJSUlNFWlBir/v7+M2fOrF69WiqVrlix4sKFC69evUpNTbVJ5UNDQyQpbFLbiFpbWymK4oyR45g5c2ZJSYlIJEpOTr53796IZfif6qSkJHK6NmzYEBMTo9VqOzs72UoeP378xRdfvPPOO56enuHh4VeuXGEYxmKP/vh4eXnRNE3OAAAgMC0SXmAWFhY+ePAgOzvbYkkEJkxoyDeLhJdvrMzMTKPR+Ndff4WFhUVGRrJdGCzkm0XoswCwo9LSUoqiYmJi2C3u7u7r169/8eKFrQaASaVSMqKMWLJkyZw5cxoaGmwSfDdu3Ojq6lq9erX1VY2GPJQoEonMF1u1alVOTk5/f39CQsKLFy+GF+B/qleuXMn+m/TWt7S0kJdlZWVTpkzZuHEjW8Df3z88PPzWrVt6vX6sh8aHm5vbiIcDMAkhMC0SWGA+efJk//79xcXFUqmUT3kEJkxcyDeLBJZvHGKxOCws7OzZs7GxsUeOHPn11185BZBv5qHPAsBejEajwWDw8PDgPKPr5+dHUVRbW5tNPsXb25uzZfbs2RRFtbe326R+e/Pw8KAoamBgwGLJlJSUrVu33rlzh7MCFjXGU23ahS8WiymKGhoaYisZGhqSyWS0id9//52iqAcPHozvAM0bHBzkOWEngLAhMPkQWGCSkdtr165l307WOj18+DB5+ffff5uWR2DCBIV840Ng+TaaTZs2URRVUVHB2Y58Mw99FgD24u7uLpPJXr582dvba7qdjEzz9/cnL6dMmfLq1SvTAt3d3ZyqaJoe7VP++ecfzlA9cnEiFyrr67e3gIAAiqLIg3wWFRUVLV68uLi4mDz0yOJ5qs1zd3f39vZ2c3MbGBgY/hzdunXr+B4Sbz09PQzDkDMAMMkhMPkQWGDu3buX80bOfBaLFi1iCyMwYeJCvvEhsHwzUzlFUV1dXaYbkW8Woc8CwI7i4uIoijJdu8hoNFZXV0skEpVKRbYEBAQ0NzezBdra2p48ecKpZ9q0aexlZvHixV999RW76+XLl1qtln35xx9/tLS0REREsMFnZf329vrrr1MUxXOgnaen5/fffy+VSs+cOcPZxedUWxQfHz84OMjOs02cPn163rx59lg0m/xcyBkAAASmRQhMBCZMUMg3i4SXb2lpaYmJiZyNlZWV1H8fS6GQbzygzwLAjrKysoKDg9VqdUVFRW9vr06n2759e2tra25uLhmlRlGUUqlsaWnJy8vr6+trbGxMTU1le8RZy5cv1+l0T58+1Wg0TU1NCoWC3SWTyQ4ePKjRaPr7++vq6hITE8VicW5uLlvAmvqjo6N9fX1v3rxp+1PzfxEREbNnz25oaOBZPjw8vKCgYPh2PqfaoqysrJCQkJ07d1ZWVhoMhq6uroKCgmPHjuXk5LArfiUmJtI0/fDhQ551mkEWzVIqldZXBSAACEyLEJgITJigkG8WCTLfLl26dOzYsUePHhmNxkePHh04cODChQtyuTwpKcm0GPLNsrEsMgLmYK3TyYbit7ZQZ2enWq0ODg4WiUQymUylUlVXV5sW6O7uTkpKCggIkEgkUVFRWq1WLpeTr+eBAwdImXv37ikUCqlUGhQUlJ+fz743IiIiMDDw7t27KpVq+vTpEolkzZo1NTU1tqpfoVD4+PjwWdjJmt//gwcPurm5NTc3k5cdHR2mGSWXy4e/Zffu3ZylrRizp1qj0ZjWeejQIea/4ydjYmJISbKm98KFC0Ui0axZs5RKZVVVlemnREdHe3p6Dg4Omjmi8vLy4WFrusgWkZCQEBgY+OrVK75nygTWOoUJhH8+TJLAtOZ7J7zAJJKTkzmZqVKpTAtYE5g8r9cOqweEBPnGgXxjGQyGoqIilUq1YMECsVjs6ekpl8uzsrL+/fdfTklXyDcXh7+xbQZ9FpONK2QEuUQ5tw2ENb//3d3dgYGBycnJtm2SPTx//lwikSQlJVlfVX19PU3Tly9fHt/b0WcBE4iLXB9dJzCt+d4hMMcBfRZgP8g3DuTbWLlIvrk4PBsCAE4mk8nKy8uvXbuWn5/v7LaYwzBMSkqKl5fX8IW1x6qpqSk+Pj4jI2Pbtm02aRsATBIITAAQKuQbjAZ9FgDgfJGRkXV1dZWVlT09Pc5uy6iePXvW1NRUXV3Nc95pMwoKCk6ePHny5EmbNAwAJhUEJgAIFfINRoQ+C4AJKScnh6bphoaG5uZmmqYzMzOd3SJrLViwoKKiwsvLy9kNGZW/v39NTU14eLj1VZ0+fRod6gAOg8B0PAQmgGMg3xwP+eZ4bs5uAACMR1paWlpamrNbAQAwASAwAUCokG8wGWCcBQAAAAAAAAC4IvRZAAAAAAAAAIArQp8FAAAAAAAAALgi9FkAAAAAAAAAgCvCHJw2VlJS4uwmgONoNBpnN8FVkFOB33+H0ev1c+fOtVVV+MGBXSEfOPR6PYUTMjHhug8cyDcO5BvYBQM2cvXqVWf/MAFgEtmyZYv1wbVlyxZnHwcAgN1dvXrV+sB09kEAAIzAJvnm4mgGEQwAAAAAAAAArgfzWQAAAAAAAACAK0KfBQAAAAAAAAC4IvRZAAAAAAAAAIArQp8FAAAAAAAAALii/wEJI1HAvfCwFQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 627 ms (started: 2021-07-27 16:35:28 +08:00)\n"
     ]
    }
   ],
   "source": [
    "keras.utils.plot_model(new_model, \"updated_ticket_classifier.png\", show_shapes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "helpful-trouble",
   "metadata": {},
   "source": [
    "### 子类化Model类"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "occasional-hobby",
   "metadata": {},
   "source": [
    "您应该了解的最后一个模型构建模式是最高级的模式：模型子类化。 您已经在第 3 章中学习了如何将类子类化以创建图层自定义图层。 子类化模型非常相似：\n",
    "* 在 init 方法中，定义模型将使用的层。\n",
    "* 在 call 方法中，定义模型的前向传递，重用先前创建的层。\n",
    "* 实例化您的子类并在数据上调用它以创建其权重。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "convertible-tooth",
   "metadata": {},
   "source": [
    "**将我们之前的示例重写为子类模型**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "incoming-verse",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.91 ms (started: 2021-07-27 16:49:37 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 译者注：这个就和Pytorch的方式极其相似了，只不过Pytorch用的Forward()函数，而不是call()\n",
    "\n",
    "class CustomerTicketModel(keras.Model):\n",
    "    \n",
    "    def __init__(self, num_departments):\n",
    "        # 不要忘记调用super构造函数！ \n",
    "        super().__init__() \n",
    "        \n",
    "        # 在构造函数中定义子层。\n",
    "        self.concat_layer = layers.Concatenate()\n",
    "        self.mixing_layer = layers.Dense(64, activation=\"relu\")\n",
    "        self.priority_scorer = layers.Dense(1, activation=\"sigmoid\") \n",
    "        self.department_classifier = layers.Dense(num_departments, activation=\"softmax\")\n",
    "    \n",
    "    # 在call方法中定义前向传递。\n",
    "    def call(self, inputs):\n",
    "        title = inputs[\"title\"]\n",
    "        text_body = inputs[\"text_body\"]\n",
    "        tags = inputs[\"tags\"]\n",
    "\n",
    "        features = self.concat_layer([title, text_body, tags])\n",
    "        features = self.mixing_layer(features)\n",
    "        priority = self.priority_scorer(features)\n",
    "        department = self.department_classifier(features)\n",
    "        return priority, department"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "conservative-campbell",
   "metadata": {},
   "source": [
    "一旦定义了模型，就可以实例化它。 请注意，它只会在您第一次在某些数据上调用它时创建它的权重——很像层子类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "closing-poster",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 205 ms (started: 2021-07-27 16:51:04 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = CustomerTicketModel(num_departments=4)\n",
    "\n",
    "priority, department = model({\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "referenced-offer",
   "metadata": {},
   "source": [
    "到目前为止，一切看起来都与图层子类化非常相似，这是您在第 3 章中已经遇到的工作流程。那么，图层子类和模型子类之间的区别是什么？ 很简单：“层”是您用来创建模型的构建块，而“模型”是您实际训练、导出以进行推理等的顶级对象。简而言之，模型具有 fit() ，evaluate() 和 predict() 方法。 层没有。 除此之外，这两个类实际上是相同的（另一个区别是您可以将模型转换为磁盘上的文件——我们将在几节中介绍保存）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pacific-large",
   "metadata": {},
   "source": [
    "您可以像 Sequential 或 Functional 模型一样编译和训练子类："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "vulnerable-hawaii",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "40/40 [==============================] - 1s 13ms/step - loss: 5.5570 - output_1_loss: 0.3218 - output_2_loss: 5.2352 - output_1_mean_absolute_error: 0.4916 - output_2_accuracy: 0.2625\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fc3d64e5670>"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.38 s (started: 2021-07-27 17:01:38 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer=\"adam\",\n",
    "              loss=[\"mean_squared_error\", \"categorical_crossentropy\"], \n",
    "              metrics=[[\"mean_absolute_error\"], [\"accuracy\"]])\n",
    "\n",
    "model.fit({\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data}, \n",
    "          [priority_data, department_data],\n",
    "          epochs=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "reasonable-effects",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "40/40 [==============================] - 1s 12ms/step - loss: 3.2431 - output_1_loss: 0.3290 - output_2_loss: 2.9140 - output_1_mean_absolute_error: 0.4992 - output_2_accuracy: 0.2125\n",
      "time: 1.85 s (started: 2021-07-27 17:02:07 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.evaluate({\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data},\n",
    "               [priority_data, department_data])\n",
    "\n",
    "priority_preds, department_preds = model.predict( {\"title\": title_data, \"text_body\": text_body_data, \"tags\": tags_data})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wireless-novel",
   "metadata": {},
   "source": [
    "您作为 and 传递的结构必须与 call() 返回的损失指标完全匹配——因为我们返回了两个元素的列表，所以 loss 和 be 两个元素的列表也应该如此。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sound-field",
   "metadata": {},
   "source": [
    "您作为损失和指标传递的结构必须与 call() 返回的内容完全匹配——因为我们返回了两个元素的列表，所以损失和指标应该是两个元素的列表。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "different-discipline",
   "metadata": {},
   "source": [
    "输入数据的结构必须与 call() 方法所期望的完全匹配，目标数据的结构必须与 call() 方法返回的完全匹配。 在这里，输入数据必须是具有 3 个键（'title'、'text_body' 和 'tags'）的 dict，目标数据必须是包含两个元素的列表。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exclusive-rates",
   "metadata": {},
   "source": [
    "模型子类化工作流是构建模型最灵活的方式：它使您能够构建无法表示为层的有向无环图的模型——例如，想象一下，call() 方法在循环中使用层的模型， 甚至递归调用它们。 因为一切皆有可能——您负责。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pressed-understanding",
   "metadata": {},
   "source": [
    "**当心：哪些子分类模型不支持**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "humanitarian-cartridge",
   "metadata": {},
   "source": [
    "这种自由是有代价的：对于子类模型，您要负责更多的模型逻辑，这意味着您的潜在错误面要大得多。 因此，您将有更多的调试工作要做。 您正在开发一个新的 Python 对象，而不仅仅是拼凑乐高积木。\n",
    "\n",
    "函数模型和子类模型在本质上也有很大的不同：功能模型是一种显式的数据结构 - 一个层图，您可以查看、检查和修改它。 同时，子类模型是一段字节代码——一个带有 call() 方法的 Python 类，其中包含原始代码。 这是子类化工作流灵活性的来源——你可以编写任何你喜欢的功能——但它引入了新的限制。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "polar-satin",
   "metadata": {},
   "source": [
    "例如，由于层相互连接的方式隐藏在 call() 方法的主体内，因此您无法访问该信息。 调用 summary() 不会显示图层连接性，并且您无法通过 plot_model() 绘制模型拓扑。 同样，如果您有一个子类模型，则无法访问层图的节点来进行特征提取——因为根本没有图。 一旦模型被实例化，它的前向传递就变成了一个完整的黑盒。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "olive-reunion",
   "metadata": {},
   "source": [
    "### 混合和匹配不同的组件"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prostate-saturday",
   "metadata": {},
   "source": [
    "至关重要的是，选择其中一种模式——顺序模型、函数API、模型子类化——并不会将你排除在其他模式之外。 Keras API 中的所有模型都可以顺畅地相互操作，无论是顺序模型、函数模型还是\n",
    "从头开始编写的子类模型。 它们都属于同一范围的工作流程。\n",
    "\n",
    "例如，您可以在功能模型中使用子类层或模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "extra-keyboard",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 115 ms (started: 2021-07-27 17:13:47 +08:00)\n"
     ]
    }
   ],
   "source": [
    "class Classifier(keras.Model):\n",
    "    def __init__(self, num_classes=2):\n",
    "        super().__init__()\n",
    "        if num_classes == 2:\n",
    "            num_units = 1\n",
    "            activation = \"sigmoid\"\n",
    "        else:\n",
    "            num_units = num_classes\n",
    "            activation = \"softmax\"\n",
    "            self.dense = layers.Dense(num_units, activation=activation)\n",
    "\n",
    "    def call(self, inputs):\n",
    "        return self.dense(inputs)\n",
    "    \n",
    "inputs = keras.Input(shape=(3,))\n",
    "\n",
    "features = layers.Dense(64, activation=\"relu\")(inputs)\n",
    "\n",
    "outputs = Classifier(num_classes=10)(features)\n",
    "\n",
    "model = keras.Model(inputs=inputs, outputs=outputs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "african-joseph",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_4\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_3 (InputLayer)         [(None, 3)]               0         \n",
      "_________________________________________________________________\n",
      "dense_15 (Dense)             (None, 64)                256       \n",
      "_________________________________________________________________\n",
      "classifier (Classifier)      (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 906\n",
      "Trainable params: 906\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "time: 2.33 ms (started: 2021-07-27 17:13:55 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "radical-fields",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOAAAAD/CAYAAADordqvAAAABmJLR0QA/wD/AP+gvaeTAAAdGklEQVR4nO3de1SUdf4H8PfcuYjDJQQSFLWM2pCK7AjJAhpoJ5VkUXMVcVtdizYzD5Vt5Xa0k5mW2cmyrI6nzU6YZ2UztdVO5G4Im5qXarmYHo8molyWEeQiMJ/fH7vMz/EZkMvA19H365z5w+985pnPPN95+1yGeUYnIgIiUkKvugGi6xkDSKQQA0ikEANIpJDx8oHCwkK8/vrrKnohuqYtXrwYcXFxTmOaLeCpU6ewZcuWfmuK6HqwZcsWnDp1SjOu2QK2++yzz/q0IaLriU6ncznOY0AihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRjAHhARFBQU4LHHHsPIkSNhsVgwaNAgjB07Fh9//DF6e52rAQMGQKfTOd1Wr17tpu7737X2etzJLQGsr6/HzTffjEmTJrljcVe90tJSjB07FmVlZdiyZQtsNhuKioowZMgQZGZm4qmnnurV8uvr63Hw4EEAQFpaGkQEOTk57mhdiWvt9biTWwIoIrDb7bDb7e5YXJ8aMGAAxo4d2+vlGI1GbN68GaNGjYKXlxeGDx+OjRs3IigoCG+99Raam5vd0K3ncNd6vd50+IXc7vDz88OxY8fcsSiPEBUVhZaWFs242WxGREQEDh06hKamJlgsFgXdkSfhMaAb1dbW4ujRo7jzzjthtVpVt0MeoNcBzMvLczq4bmpqcjl+4sQJzJgxA/7+/ggKCsKkSZOctpqrV6921IaHh2Pfvn0YP348/Pz84OPjg+TkZBQUFDjqX3rpJUf9pbs+X375pWP8hhtu0Cz/woULKCgocNQYjb3fCTh//jwKCgowZcoUhIaG4qOPPur1MjtyvazX1tZW5ObmIiUlBaGhofD29kZ0dDTWrl3rONSpra3VnNx56aWXHI+/dDwjI8Ox7MrKSixcuBCRkZEwm80IDg5Geno6Dh061OF6Li0txfTp0xEUFOQYq6qq6tVrBADIZXJzc8XF8BWlpaUJAGlsbHQ5npaWJnv37pX6+nrZvXu3eHt7y+jRozXLiYmJEV9fX4mLi3PU79u3T0aNGiVms1m++eYbp3pfX1+59957NcuJjY2VoKAgzXhH9T21fPlyASAAJCkpSY4cOeKyLjk5WQIDA6WwsLBLyz148KBjvbniaev1Sq/nctu2bRMA8vLLL0tNTY1UVlbKm2++KXq9XnJycpxqJ0yYIHq9Xn7++WfNcuLi4mTTpk2Of5eXl8vQoUMlJCREtm/fLnV1dfLjjz9KYmKieHl5yd69e50e376eExMTJT8/Xy5cuCBFRUViMBiksrKyS69FRASA5ObmascvH+irAG7bts1pPCMjQwBoXkRMTIwAkIMHDzqNHzlyRABITEyM07jqAIqINDc3S3FxsTzyyCNiMBhk2bJlmprExEQJCAjQTHBHuhpAT1mvPQlgUlKSZnz27NliMpnEZrM5xv7+978LAMnOznaq/fbbb2Xw4MFy8eJFx1hWVpYAcAqliMiZM2fEYrFIbGys03j7et6xY0eX+u5IRwHst2PA0aNHO/07IiICAFBeXq6p9fX1xR133OE0Fh0djRtvvBGHDx/GmTNn+q7RHjCbzYiKisI777yDKVOmYOnSpfjqq6+car755hvU1NRorgvZW9fqep00aRLy8/M14zExMWhpacFPP/3kGEtNTUV0dDQ2btyI6upqx/iqVavw+OOPw2QyOcby8vKg1+s1H5mFhobiV7/6FQ4cOIBffvlF87z33HOPO16WRr8F8PKTEmazGQBcfnTh7+/vchmDBg0CAJw7d87N3bnP5MmTAQBffPFFvzzftbpebTYbli5diujoaAQEBDiOu9o/Y21oaHCqX7RoERoaGvD2228DAMrKyvD111/jD3/4g6OmubkZNpsNdrsdVqtVc/z4/fffAwCOHj2q6cfX17dPXudVeRa0urra5V+TtL9B2t8wAKDX63Hx4kVNbW1trctld3R9Rndp/+ihpqamT5+nJzxpvU6ePBnLly/H/PnzUVZWBrvdDhHBmjVrAEDzOmbNmoWQkBDHZ7CvvfYasrKyEBAQ4KixWCzw9/eH0WhES0sL5L+HYJpbcnKyW19LZ67KADY1NWHfvn1OYz/88APKy8sRExODsLAwx3hYWBhOnz7tVFtRUYGTJ0+6XLaPj4/TG+uWW27Be++9163+cnJyMHv2bJf37dy5E4B21/BqcLWvV6PRiJKSErS1taGgoAChoaFYuHAhgoODHQFvbGx0+ViLxYLs7GycO3cOr732GjZt2oQnnnhCU5eeno7W1lanM7/tVq5ciSFDhqC1tbVbfffGVRlAq9WKP/3pTygsLMSFCxewf/9+zJ49G2azGWvXrnWqTU1NRXl5Od566y3U19fj2LFjeOKJJ5z+N7/UXXfdhbKyMpw6dQqFhYU4fvw4EhISut3jJ598gmXLluHEiRNobm7GiRMn8Mwzz+Djjz9GbGws5s2b51Q/btw4BAUFoaioqNvP5S6esF4BwGAwICkpCRUVFVi1ahWqqqrQ2NiI/Px8rF+/vsPHZWdnw9vbG88//zzuu+8+3HTTTZqaFStWYMSIEXj44Yexc+dO2Gw21NTU4N1338WyZcuwevVqt3w01WWXn5Xp7lnQrVu3Ok7Dt99mzZolhYWFmvHnnnvOcUbo0tsDDzzgWF5MTIwMHjxY/v3vf8uECRPEz89PvL29JTExUb799lvN89fW1sq8efMkLCxMvL29ZezYsbJv3z6JjY11LP+ZZ55x1JeUlEhCQoL4+vpKRESErFu3rsuvtZ3NZpP3339fJkyYIJGRkWI2m2XAgAESGxsrK1askIaGBs1jEhISunwW1NfXV7OOVq1aJSLikevV1evp6FZcXCwiIpWVlbJgwQKJiIgQk8kkISEhMnfuXFmyZImj9vIzliIi8+fPFwCyZ8+eDtdvdXW1LF68WIYPHy4mk0mCg4MlNTVVdu/e7ahxtZ67k4vLoa8/hnCX9jcKudf1sl4//PBDl8FUraMAXpW7oEQ9tX79eixevFh1G13GAJJHe//99zF16lTU19dj/fr1+M9//oPp06erbqvLrpoAtv9N4eHDh3H69GnodDo8//zz/fb8l38m5Or24osv9ls/7qJ6vfaHvLw8BAQE4J133sGnn37avydRekn3v/1Th82bN2PGjBm9/lY3Ef0/nU6H3Nxczdb5qtkCEl2PGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFOvzexrRp0/qzD6LrkmYLGBER4XQdfbp67d+/H/v371fdBnVBRkaG46LJl9J8H5A8R/t3yzZv3qy4E+opHgMSKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxAASKcQAEinEABIpxF/I9RAbN27EG2+8gba2NsdYZWUlACA4ONgxZjAYsGjRIsydO7e/W6QeYAA9RGlpKaKiorpUW1xc3OVaUou7oB7illtuQXR0NHQ6XYc1Op0O0dHRDJ8HYQA9yJw5c2AwGDq832g0Iisrqx87ot7iLqgHKS8vR3h4ODqaMp1Oh5MnTyI8PLyfO6Oe4hbQg9x4442Ij4+HXq+dNr1ej/j4eIbPwzCAHiYzM9PlcaBOp8OcOXMUdES9wV1QD1NTU4OQkBC0trY6jRsMBpw9exZBQUGKOqOe4BbQwwQGBiIlJQVGo9ExZjAYkJKSwvB5IAbQA82ePRt2u93xbxFBZmamwo6op7gL6oEuXLiAG264AU1NTQAAi8WCqqoqDBgwQHFn1F3cAnogX19fTJkyBSaTCUajEQ8++CDD56EYQA81a9YstLa2oq2tDb/97W9Vt0M9ZLxySfds3rzZ3YskF9ra2uDl5QURQX19Pdd7P5k+fbpbl+f2Y8DO/laRyNO5+5SJ27eAAJCbm+v2/ylIKz8/HzqdDklJSapbueZt3rwZM2bMcPty+ySA1D8SExNVt0C9xAB6MFd/E0qehTNIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKTQVRnATz/9FDqdDjqdDl5eXqrb6Xc7duzAyJEjnS685MrYsWMd6+ny26JFi3rVw4ABAzTL1Ov1CAgIQExMDLKzs3HgwIFePQddpQF86KGHICIYP3686lb61bFjxzBlyhQ8++yzOHv2rNJe6uvrcfDgQQBAWloaRAQtLS0oKSnBsmXLUFJSgrvvvhu/+93v0NDQoLRXT3ZVBvB69cILLyA+Ph4HDhyAn59flx6zb98+iIjm9sYbb7i9P4PBgJCQEKSlpeHrr7/G008/jY0bN2LmzJlu/6Lq9YJfR7qKfPDBB/D29lbdRpe98sor2LNnDz7//HN8+umnmDlzpuqWPA63gFcRTwof8N/Lj/zxj38EALz99tuKu/FMV0UAS0pK8OCDD8JqtcLX1xcJCQn49ttvO6yvrKzEwoULERkZCbPZjODgYKSnp+PQoUOOmry8PKcTCCdOnMCMGTPg7++PoKAgTJo0CceOHXNabnNzM5YuXYqoqCj4+PggMDAQkydPxueff+70y7Rd7aE//OUvf8Edd9wBX19fWK1WJCQk4JNPPum35x87diwAoKioCC0tLY5xzlEXiZsBkNzc3C7XHz16VPz9/WXw4MGya9cuqaurkyNHjkhqaqpERkaKxWJxqi8vL5ehQ4dKSEiIbN++Xerq6uTHH3+UxMRE8fLykr179zrVp6WlCQBJS0uTvXv3Sn19vezevVu8vb1l9OjRTrXz5s0Tq9Uqu3btkoaGBqmoqJCcnBwBIPn5+T3uoScGDx4sBoOh05p7771XMjMz5cCBA1JfXy8lJSWSmZkpAOTxxx/X1CcnJ0tgYKAUFhZ2qYeDBw861l1HGhsbBYAAkPLychG5NucoNzdX+iAuojyA06ZNEwCyZcsWp/HTp0+LxWLRBDArK0sAyKZNm5zGz5w5IxaLRWJjY53G2yd327ZtTuMZGRkCQCorKx1jw4YNk/j4eE2PI0eOdJrc7vbQE10JYEfuueceASBFRUVO44mJiRIQENDlN19XAtjQ0KAJ4LU4R9dsAP38/ASA1NXVae6Ljo7WBNBqtYperxebzaapv+uuuwSAnDp1yjHWPrkVFRVOtU8++aQAkMOHDzvGHn30UQEg8+fPl8LCQmltbXXZc3d76IneBPDVV18VAPLcc8/1qoeuBPDYsWMCQEwmk1y8eFFErs056qsAKj0GbG5uRl1dHby8vFxeWn3QoEGaepvNBrvdDqvVqvmg+PvvvwcAHD16VLMsq9Xq9G+z2QwATj9ysm7dOnz00Uc4fvw4xo8fj4EDB2LixInYunWrW3roL2FhYQCAc+fO9flztR+rx8XFwWQycY66SWkALRYL/Pz80NTUhPr6es39NTU1mnp/f38YjUa0tLS4/PxLRJCcnNyjfnQ6HTIzM/HVV1+htrYWeXl5EBGkp6fj9ddf75ce3KG8vByA9j8wd7Pb7Vi3bh0A4LHHHgPAOeou5WdB77//fgDAl19+6TReVVWF0tJSTX16ejpaW1tRUFCguW/lypUYMmSI5scru8rf3x8lJSUAAJPJhJSUFMeZuu3bt/dLD131/vvvIzY2VjMuIo7L1E+ePLlPe3j22Wfx3XffYerUqZg2bZpjnHPUDe7ep0U3jwF//vlnCQwMdDoL+tNPP8mECRNk0KBBmmPAs2fPyogRI2T48OGyY8cOqa2tlerqalm/fr34+Phonrv9+KKxsdFp/JlnnhEAcvDgQceY1WqVxMREOXz4sDQ1NcnZs2flxRdfFADy0ksv9biHnrjSMeCGDRsEgGRnZ8vRo0elsbFRSkpKZNasWX12FrStrU3Onj0reXl5Mm7cOAEgDz/8sDQ0NDg97lqco2v2JIyISGlpqTz44IMycOBAx6nnL774QsaPH+84w/b73//eUV9dXS2LFy+W4cOHi8lkkuDgYElNTZXdu3c7agoLCx2Pbb+1n5S4fPyBBx4QEZFDhw7JggUL5NZbbxUfHx8JDAyUMWPGyIYNG8Rutzv13JUeumvbtm2a3tpvGzZscKptamqSzz77TKZOnSojRowQi8UiVqtVkpKS5JNPPnG5/ISEhC6fBfX19dX0oNPpxGq1SnR0tDz66KNy4MCBDh9/rc1RXwWwT36chb8NQdea9t+GcHNc1B8DEl3PGEAihRjAPtTRl2Uvvb344ouq2ySF+HWkPuTu4wW69nALSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKRQn3wborCwsC8WS6RMX72n++SSFETXKnd/xcztW0B+B67/tF93p/0yhOR5eAxIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikEANIpBADSKQQA0ikUJ/8Rjy53549e1BUVOQ0VlJSAgBYuXKl0/iYMWOQmJjYb71Rz7n9N+Kpb+zevRupqakwmUzQ613vuNjtdrS0tGDXrl1ISUnp5w6pJxhAD9HW1oaQkBBUV1d3WhcQEIBz587BaOTOjSfgMaCHMBgMmDVrFsxmc4c1ZrMZmZmZDJ8HYQA9yMyZM3Hx4sUO77948SJmzpzZjx1Rb3EX1MMMHToUJ0+edHlfeHg4Tp48CZ1O189dUU9xC+hhZs+eDZPJpBk3m83Iyspi+DwMt4Aepri4GLfddpvL+3744Qfcfvvt/dwR9QYD6IFuu+02FBcXO41FRUVpxujqx11QDzRnzhyn3VCTyYSsrCyFHVFPcQvogU6ePInIyEi0T51Op8Px48cRGRmptjHqNm4BPdCQIUNw9913Q6/XQ6fTYfTo0Qyfh2IAPdScOXOg1+thMBiQmZmpuh3qIe6CeqjKykqEhYUBAE6fPo2QkBDFHVGPiAsZGRkCgDfeeHPTLSMjw1XUpMM/GhwzZgyefPLJju6mq8CePXug0+nw61//WnUr1Ik1a9Z0eF+HAQwPD8f06dP7pCFyj4kTJwIABg4cqLgT6sxnn33W4X38s3kPxuB5Pp4FJVKIASRSiAEkUogBJFKIASRSiAEkUogBJFKIASRSiAEkUogBJFKIASRSiAEkUqjfA7h69WrodDrodDqEh4f399P3WHFxMWbMmIHQ0FAYjUbHa/D391fdWqf27duHuXPnYtiwYfD29kZgYCBuv/12/OY3v8E777yDY8eOOWo5N/2v3wOYk5MDEUFMTEx/P3Wn6uvrcfPNN2PSpEma+06cOIG4uDgUFxfjr3/9K86fP4/z589j8+bNHf5SkWp2ux1PPfUU4uPjMWjQIOzcuRO1tbUoLi7GmjVrcP78eWRnZ+Omm25Ca2srAM6NCld/h/1ERGC322G32zX3vffee7DZbFi3bh3i4+Ph4+MDPz8/TJs2DTU1NQq6vbIXXngBq1evxttvv41XX30VUVFRsFgsCAkJQUpKCr788kvcf//9qtvskmttbi7F7wP+j5+fn9Pu2KWOHj0KABg1alR/ttRjJSUleOWVVxAbG4v58+e7rDEYDHjhhRewc+fOfu6u+66lubkcA9gFLS0tAACLxaK4k6557733YLfbMW3atE7r4uLiHNcW9VSeNjeXc+suaHV1NRYvXowRI0bAYrEgPDwc9913HzZu3IjGxsYrPr61tRW5ublISUlBaGgovL29ER0djbVr12p2P5qbm7F06VJERUXBx8cHgYGBmDx5Mj7//HO0tbV1qy4vL89x4K7T6dDU1OQ0/re//Q0A4O3t7VTXfps7d67mtVRWVmLhwoWIjIyE2WxGcHAw0tPTcejQIUfN5c9bWlqK6dOnIygoyDFWVVXV7Xn4xz/+AcC9WwXOjXvmRqOjq6J1dBWnjpw5c0aGDRsmoaGhsm3bNjl//rxUVFTI8uXLBYCsWbPGqT4mJkYGDx7sNLZt2zYBIC+//LLU1NRIZWWlvPnmm6LX6yUnJ8epdt68eWK1WmXXrl3S0NAgFRUVkpOTIwAkPz+/23UiImlpaQJAGhsbuzReWVkpACQrK8tpvLy8XIYOHSohISGyfft2qaurkx9//FESExPFy8tL9u7d63L5iYmJkp+fLxcuXJCioiIxGAxSWVkpIiLJyckSGBgohYWFnc6DiEhYWJgAkH/9619XrHWFc6NdfmdzcyWd5cltAZw7d64AkNzcXM19EydO7HIAk5KSNI+fPXu2mEwmsdlsjrFhw4ZJfHy8pnbkyJFOk9fVOhH3TXJWVpYAkE2bNjmNnzlzRiwWi8TGxrpc/o4dOzR9tktMTJSAgADNG8SV9gB+9913V6x1hXOjXX5nc3Ml/RJAq9UqAOT8+fNdqnc1yR1ZtWqVAHB68z366KMCQObPny+FhYXS2trq8rFdrRNx3yRbrVbR6/VOb8p2d911lwCQU6dOaZZfVVXVYW/dERsb26s3DefGvXPTWZ7ccgzY3NwMm80GLy8v+Pn59Xg5NpsNS5cuRXR0NAICAhz72k899RQAoKGhwVG7bt06fPTRRzh+/DjGjx+PgQMHYuLEidi6davTMrta5y7t68Jut8NqtWqOSb7//nsA/3/27lK+vr5u6SExMREAcOTIEbcsD+DcuGtuLueWAFosFlitVjQ1NaGurq7Hy5k8eTKWL1+O+fPno6ysDHa7HSLiuLCpXHLGTqfTITMzE1999RVqa2uRl5cHEUF6ejpef/31bte5i8Vigb+/P4xGI1paWiD/3cvQ3JKTk93+3O0WLFgAo9GILVu2dFr39NNPQ6/Xo6Sk5IrL5Nz0DbedBZ06dSoAYMeOHZr77rzzziteZbutrQ0FBQUIDQ3FwoULERwc7Pi5ZVdnUP39/R1vHJPJhJSUFMeZq+3bt3e7zp3S09PR2tqKgoICzX0rV67EkCFDHH990hdGjhyJP//5z9i/fz8+/PBDlzWlpaV49913MX36dERFRXW6PM5N33FbAFesWIFhw4bhySefxPbt21FXV4dffvkF2dnZOHPmzBUDaDAYkJSUhIqKCqxatQpVVVVobGxEfn4+1q9f7/IxjzzyCI4cOYLm5macO3cOr776KkQE48aN61Gdu6xYsQIjRozAww8/jJ07d8Jms6Gmpgbvvvsuli1bhtWrV8No7N5HsOPGjUNQUBCKioq6VP/8889jyZIleOSRR7BkyRKUlZXh4sWLOH36ND744AMkJydj1KhR+OCDD664LM5NH+ruQWNnqqqqZNGiRTJs2DAxmUwSFhYmDz30kJSVlTlq2g/aL70999xzIvLfA+cFCxZIRESEmEwmCQkJkblz58qSJUscte1nqQ4dOiQLFiyQW2+9VXx8fCQwMFDGjBkjGzZsELvd7ni+rtRt3bpV09OsWbM6HBcRmTBhgua+f/7zn47nra6ulsWLF8vw4cPFZDJJcHCwpKamyu7dux01hYWFLn/Iw5WEhIQunwW91HfffSeZmZmOdern5ydjxoyRtWvXSnNzs1Mt56Znc3MlneXJ5c+Ttf8FRWfXtCeiruksT/xjbCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihRhAIoUYQCKFGEAihTq8+MWWLVscF94hot7JyMhwOe7ykhSFhYU4depUnzdFdL2IiIhAXFycZtxlAImof/AYkEghBpBIIQaQSCEjAF78k0iR/wMlVptAnlPvCAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 512 ms (started: 2021-07-27 17:14:00 +08:00)\n"
     ]
    }
   ],
   "source": [
    "plot_model(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "optical-theory",
   "metadata": {},
   "source": [
    "相反，您可以使用函数模型作为子类层或模型的一部分："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "everyday-motor",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 31.7 ms (started: 2021-07-27 17:16:53 +08:00)\n"
     ]
    }
   ],
   "source": [
    "inputs = keras.Input(shape=(64,))\n",
    "\n",
    "outputs = layers.Dense(1, activation=\"sigmoid\")(inputs)\n",
    "\n",
    "binary_classifier = keras.Model(inputs=inputs, outputs=outputs)\n",
    "\n",
    "class MyModel(keras.Model):\n",
    "    def __init__(self, num_classes=2):\n",
    "        super().__init__()\n",
    "        self.dense = layers.Dense(64, activation=\"relu\")\n",
    "        self.classifier = binary_classifier\n",
    "    def call(self, inputs):\n",
    "        features = self.dense(inputs)\n",
    "        return self.classifier(features)\n",
    "    \n",
    "model = MyModel()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "statistical-object",
   "metadata": {},
   "source": [
    "### 记住：为工作使用正确的工具"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "personalized-commodity",
   "metadata": {},
   "source": [
    "您已经了解了构建 Keras 模型的一系列工作流程，从最简单的工作流程（顺序模型）到最高级的工作流程（模型子类化）。什么时候应该使用一个？每一种都有其优点和缺点——选择最适合手头工作的一种。\n",
    "\n",
    "通常，Functional API 为您提供了易用性和灵活性之间的很好的权衡。它还使您可以直接访问图层连接，这对于模型绘图或特征提取等用例非常强大。如果你使用函数式 API——也就是说，如果你的模型可以表示为层的有向无环图——我建议使用它而不是模型子类化。\n",
    "\n",
    "展望未来，本书中的所有示例都将使用函数式 API——这仅仅是因为我们将使用的所有模型都可以表示为层图。然而，我们将频繁使用子类层。一般而言，使用包含子类层的功能模型可提供两全其美的好处：高开发灵活性，同时保留函数 API 的优势。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "educational-aerospace",
   "metadata": {},
   "source": [
    "## 使用内置的训练和评估循环"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "deadly-joint",
   "metadata": {},
   "source": [
    "逐步公开复杂性的原则——访问从简单到任意灵活的一系列工作流程，一次一个步骤——也适用于模型训练。 Keras 为您提供了不同的训练模型工作流程——它可以像对数据调用 fit() 一样简单，也可以像从头开始编写新的训练算法一样高级。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proud-dividend",
   "metadata": {},
   "source": [
    "您已经熟悉 compile()、fit()、evaluate()、predict() 工作流程。 提醒一下，它看起来像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "systematic-virus",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "1563/1563 [==============================] - 20s 10ms/step - loss: 0.2978 - accuracy: 0.9109 - val_loss: 0.1543 - val_accuracy: 0.9536\n",
      "Epoch 2/3\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1616 - accuracy: 0.9537 - val_loss: 0.1177 - val_accuracy: 0.9698\n",
      "Epoch 3/3\n",
      "1563/1563 [==============================] - 16s 11ms/step - loss: 0.1393 - accuracy: 0.9614 - val_loss: 0.1187 - val_accuracy: 0.9684\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f8468056be0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 56.2 s (started: 2021-07-28 20:38:59 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import mnist\n",
    "\n",
    "# 创建一个模型（我们将其分解为一个单独的函数，以便以后重用）。\n",
    "def get_mnist_model(): \n",
    "    inputs = keras.Input(shape=(28 * 28,))\n",
    "    features = layers.Dense(512, activation=\"relu\")(inputs)\n",
    "    features = layers.Dropout(0.5)(features)\n",
    "    outputs = layers.Dense(10, activation=\"softmax\")(features)\n",
    "    model = keras.Model(inputs, outputs)\n",
    "    return model\n",
    "\n",
    "# 加载您的数据，保留一些用于验证。\n",
    "(images, labels), (test_images, test_labels) = mnist.load_data() \n",
    "images = images.reshape((60000, 28 * 28)).astype(\"float32\") / 255\n",
    "test_images = test_images.reshape((10000, 28 * 28)).astype(\"float32\") / 255\n",
    "train_images, val_images = images[10000:], images[:10000]\n",
    "train_labels, val_labels = labels[10000:], labels[:10000]\n",
    "\n",
    "model = get_mnist_model()\n",
    "\n",
    "# 通过指定优化器、要最小化的损失函数和要监控的指标来编译模型。\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"sparse_categorical_crossentropy\", \n",
    "              metrics=[\"accuracy\"])\n",
    "\n",
    "# 用于训练模型，可选择提供验证数据以监控 fit() 对未见数据的性能。\n",
    "model.fit(train_images, train_labels,epochs=3, validation_data=(val_images, val_labels)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "suited-attitude",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 2s 7ms/step - loss: 0.1113 - accuracy: 0.9726\n",
      "time: 2.43 s (started: 2021-07-27 17:37:46 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 使用evaluate() 计算新数据的损失和度量。\n",
    "test_metrics = model.evaluate(test_images, test_labels) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "statewide-brook",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 959 ms (started: 2021-07-27 17:37:48 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 使用 predict() 计算新数据的分类概率。\n",
    "predictions = model.predict(test_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "obvious-fence",
   "metadata": {},
   "source": [
    "有几种方法可以自定义这个简单的工作流程：\n",
    "* 提供您自己的自定义指标\n",
    "* 将回调传递给 fit() 方法以安排在训练期间的特定时间点采取的行动"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sealed-implementation",
   "metadata": {},
   "source": [
    "让我们来看看这些。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "preceding-apple",
   "metadata": {},
   "source": [
    "### 编写自己的指标"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "magnetic-holiday",
   "metadata": {},
   "source": [
    "指标是衡量模型性能的关键——尤其是衡量模型在训练数据上的性能与其在测试数据上的性能之间的差异。 用于分类和回归的常用指标已经是内置 keras.metrics 模块的一部分——大多数时候，这就是你将使用的。 但是，如果您正在做任何不寻常的事情，您将需要能够编写自己的指标。 这很简单！"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pregnant-beach",
   "metadata": {},
   "source": [
    "Keras 指标是 keras.metrics.Metric 类的子类。 与层类似，度量具有存储在 TensorFlow 变量中的内部状态。 与层不同，这些变量不会通过反向传播更新，因此您必须自己编写状态更新逻辑——这发生在 update_state() 方法中。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "considered-fluid",
   "metadata": {},
   "source": [
    "例如，这是一个测量均方根误差 (RMSE) 的简单自定义指标。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "typical-irrigation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.7 ms (started: 2021-07-28 06:28:08 +08:00)\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "# 对类进行子类化。\n",
    "class RootMeanSquaredError(keras.metrics.Metric): \n",
    "    \n",
    "#     在构造函数中定义状态变量。 与layers一样，您可以访问 add_weight() 方法。\n",
    "    def __init__(self, name=\"rmse\", **kwargs): \n",
    "        super().__init__(name=name, **kwargs)\n",
    "        self.mse_sum = self.add_weight(name=\"mse_sum\", initializer=\"zeros\")\n",
    "        self.total_samples = self.add_weight(name=\"total_samples\", \n",
    "                                             initializer=\"zeros\", \n",
    "                                             dtype=\"int32\")\n",
    "#     在 update_state() 中实现状态更新逻辑。 y_true 参数是一批的目标（或标签），而代表来自模型的相应 y_pred 预测。 \n",
    "#     为了匹配我们的 MNIST 模型，我们需要分类预测和整数标签。 您可以忽略 sample_weight 参数，我们不会在这里使用它。    \n",
    "    def update_state(self, y_true, y_pred, sample_weight=None): \n",
    "        y_true = tf.one_hot(y_true, depth=tf.shape(y_pred)[1]) \n",
    "        mse = tf.reduce_sum(tf.square(y_true - y_pred))\n",
    "        self.mse_sum.assign_add(mse)\n",
    "        num_samples = tf.shape(y_pred)[0]\n",
    "        self.total_samples.assign_add(num_samples)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sexual-failure",
   "metadata": {},
   "source": [
    "您使用 result() 方法返回指标的当前值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "affected-thermal",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.02 ms (started: 2021-07-28 06:28:23 +08:00)\n"
     ]
    }
   ],
   "source": [
    "def result(self):\n",
    "    return tf.sqrt(self.mse_sum / tf.cast(self.total_samples, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "scheduled-atlanta",
   "metadata": {},
   "source": [
    "同时，您还需要公开一种无需重新实例化即可重置度量状态的方法——这使得相同的度量对象能够在不同的训练时期或在训练和评估中使用。 你这样做是 reset_states() 方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "figured-reporter",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 509 µs (started: 2021-07-28 06:29:48 +08:00)\n"
     ]
    }
   ],
   "source": [
    "def reset_states(self):\n",
    "    self.mse_sum.assign(0.)\n",
    "    self.total_samples.assign(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "invalid-skill",
   "metadata": {},
   "source": [
    "自定义指标可以像内置指标一样使用。 让我们测试一下我们自己的指标："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "moved-national",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import layers\n",
    "\n",
    "model = get_mnist_model()\n",
    "\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\", RootMeanSquaredError()])\n",
    "model.fit(train_images, train_labels, epochs=3,validation_data=(val_images, val_labels))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "naval-duration",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_metrics = model.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "active-therapist",
   "metadata": {},
   "source": [
    "您现在可以看到进度条显示模型的 RMSE。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mobile-technician",
   "metadata": {},
   "source": [
    "### 使用回调函数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "numerical-matrix",
   "metadata": {},
   "source": [
    "使用 model.fit() 在大型数据集上运行数十个 epoch 的训练运行可能有点像发射纸飞机：经过最初的冲动，您无法控制其轨迹或着陆点。 如果你想避免糟糕的结果（从而浪费纸飞机），最好不要使用纸飞机，而是使用能够感知其环境、将数据发送回其操作员并根据其当前状态自动做出转向决策的无人机 . Keras API 将帮助您将您对回调 model.fit() 的调用从纸飞机转换为可以自我反省并动态采取行动的智能自主无人机。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adjustable-investment",
   "metadata": {},
   "source": [
    "回调是一个对象（一个实现特定方法的类实例），它在调用 fit() 时传递给模型，并在训练期间由模型在不同时间点调用。 它可以访问有关模型状态及其性能的所有可用数据，并且可以采取行动：中断训练、保存模型、加载不同的权重集或以其他方式改变模型的状态。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "norwegian-credits",
   "metadata": {},
   "source": [
    "以下是您可以使用回调的一些方法示例：\n",
    "* 模型检查点——在训练期间的不同点保存模型的当前状态。\n",
    "* 提前停止——当验证损失不再改善时中断训练（当然，保存训练期间获得的最佳模型）。\n",
    "* 在训练过程中动态调整某些参数的值——比如优化器的学习率。\n",
    "* 在训练期间记录训练和验证指标，或者在模型更新时可视化学习的表示——您熟悉的 fit() 进度条实际上是一个回调！"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "herbal-desert",
   "metadata": {},
   "source": [
    "keras.callbacks 模块包括许多内置回调（这不是一个详尽的列表）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "pregnant-designer",
   "metadata": {},
   "outputs": [],
   "source": [
    "keras.callbacks.ModelCheckpoint\n",
    "keras.callbacks.EarlyStopping\n",
    "keras.callbacks.LearningRateScheduler\n",
    "keras.callbacks.ReduceLROnPlateau\n",
    "keras.callbacks.CSVLogger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "important-sharing",
   "metadata": {},
   "source": [
    "让我们回顾一下其中的两个，让您了解如何使用它们：EarlyStopping 和 ModelCheckpoint。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aware-dayton",
   "metadata": {},
   "source": [
    "**早期停止模型检查点和回调**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "other-magnet",
   "metadata": {},
   "source": [
    "在训练模型时，很多事情一开始就无法预测。 特别是，您无法确定需要多少个 epoch 才能达到最佳验证损失。 到目前为止，我们的示例采用了训练足够多的 epoch 的策略，您开始过度拟合，使用第一次运行来确定要训练的合适的 epoch 数，然后最后使用这个最佳数从头开始新的训练运行。 当然，这种方法是浪费的。 处理此问题的更好方法是在您测量到验证损失不再改善时停止训练。 这可以使用 EarlyStopping 回调来实现。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "featured-mouse",
   "metadata": {},
   "source": [
    "一旦被监控的目标指标在固定数量的 epoch 内停止改进，EarlyStopping 回调就会中断训练。 例如，此回调允许您在开始过度拟合时立即中断训练，从而避免必须重新训练模型以减少 epoch 数。 此回调通常与 ModelCheckpoint 结合使用，它使您可以在训练期间不断保存模型（并且，可以选择仅保存当前的最佳模型：在 epoch 结束时实现最佳性能的模型版本） ："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "sacred-cuisine",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.2929 - accuracy: 0.9136 - val_loss: 0.1616 - val_accuracy: 0.9565\n",
      "Epoch 2/10\n",
      "1563/1563 [==============================] - 15s 9ms/step - loss: 0.1652 - accuracy: 0.9532 - val_loss: 0.1200 - val_accuracy: 0.9666\n",
      "Epoch 3/10\n",
      "1563/1563 [==============================] - 15s 9ms/step - loss: 0.1369 - accuracy: 0.9636 - val_loss: 0.1197 - val_accuracy: 0.9700\n",
      "Epoch 4/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.1279 - accuracy: 0.9661 - val_loss: 0.1104 - val_accuracy: 0.9749\n",
      "Epoch 5/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.1155 - accuracy: 0.9704 - val_loss: 0.1023 - val_accuracy: 0.9765\n",
      "Epoch 6/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.1105 - accuracy: 0.9732 - val_loss: 0.1018 - val_accuracy: 0.9775\n",
      "Epoch 7/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.1064 - accuracy: 0.9750 - val_loss: 0.1053 - val_accuracy: 0.9787\n",
      "Epoch 8/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.1010 - accuracy: 0.9764 - val_loss: 0.1127 - val_accuracy: 0.9773\n",
      "Epoch 9/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.0958 - accuracy: 0.9774 - val_loss: 0.1178 - val_accuracy: 0.9767\n",
      "Epoch 10/10\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.0931 - accuracy: 0.9789 - val_loss: 0.1282 - val_accuracy: 0.9752\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fae0ca81220>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2min 26s (started: 2021-07-28 06:45:56 +08:00)\n"
     ]
    }
   ],
   "source": [
    "# 回调通过 fit() 中的参数传递给模型，该参数包含一个回调列表。 您可以传递任意数量的回调。\n",
    "callbacks_list = [\n",
    "#     当改进停止时中断训练\n",
    "    keras.callbacks.EarlyStopping(monitor=\"accuracy\", #监控模型的验证准确性\n",
    "                                  patience=1,),#当准确性停止提高超过一个时期（即两个时期）时中断训练\n",
    "    #     在每个 epoch 后保存当前权重\n",
    "    keras.callbacks.ModelCheckpoint(filepath=\"checkpoint_path.keras\", #目标模型文件的路径\n",
    "                                    monitor=\"val_loss\", #这两个参数意味着除非 val_loss 有所改进，\n",
    "                                    save_best_only=True,)]#否则您不会覆盖模型文件，这使您可以在训练期间保持最佳模型。\n",
    "\n",
    "model = get_mnist_model()\n",
    "\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"]) #您监控准确性，因此它应该是模型指标的一部分。\n",
    "\n",
    "# 请注意，由于回调将监视验证丢失和验证准确性，因此您需要将 validation_data 传递给对 fit() 的调用。\n",
    "model.fit(train_images, \n",
    "          train_labels,\n",
    "          epochs=10,\n",
    "          callbacks=callbacks_list,\n",
    "          validation_data=(val_images, val_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "superior-honolulu",
   "metadata": {},
   "source": [
    "请注意，您也可以在训练后手动保存模型——只需调用 model.save('my_checkpoint_path')。 要重新加载您保存的模型，只需使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "conceptual-builder",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 147 ms (started: 2021-07-28 06:54:14 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = keras.models.load_model(\"checkpoint_path.keras\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ambient-shopping",
   "metadata": {},
   "source": [
    "### 编写自己的回调函数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "realistic-borough",
   "metadata": {},
   "source": [
    "如果您需要在训练期间执行其中一个内置回调未涵盖的特定操作，您可以编写自己的回调。 回调是通过继承类 `keras.callbacks.Callback` 来实现的。 然后，您可以实现以下任意数量的透明命名方法，这些方法在训练期间的不同点被调用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "tested-valentine",
   "metadata": {},
   "outputs": [],
   "source": [
    "on_epoch_begin(epoch, logs) #在每个周期开始时调用\n",
    "on_epoch_end(epoch, logs) #在每个周期结束时调用\n",
    "on_batch_begin(batch, logs) #在处理每批之前调用\n",
    "on_batch_end(batch, logs) #在处理每批之后调用\n",
    "on_train_begin(logs) #在训练开始时调用\n",
    "on_train_end(logs) #在训练结束时调用"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "entire-young",
   "metadata": {},
   "source": [
    "这些方法都使用一个参数调用，该参数是一个字典，其中包含有关前一个批次、时期或训练运行的信息日志：训练和验证指标等。 on_epoch_* 和 on_batch_* 方法也将周期或批次索引作为第一个参数（一个\n",
    "整数）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "talented-louisiana",
   "metadata": {},
   "source": [
    "这是一个简单的示例，在训练期间保存每批损失值的列表，并在每个 epoch 结束时保存这些值的图表："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "golden-kitty",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 771 ms (started: 2021-07-28 10:55:04 +08:00)\n"
     ]
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "\n",
    "class LossHistory(keras.callbacks.Callback):\n",
    "    def on_train_begin(self, logs):\n",
    "        self.per_batch_losses = []\n",
    "\n",
    "    def on_batch_end(self, batch, logs):\n",
    "        self.per_batch_losses.append(logs.get(\"loss\"))\n",
    "        \n",
    "    def on_epoch_end(self, epoch, logs):\n",
    "        plt.clf()\n",
    "        plt.plot(range(len(self.per_batch_losses)), \n",
    "                 self.per_batch_losses,\n",
    "                 label=\"Training loss for each batch\")\n",
    "        plt.xlabel(f\"Batch (epoch {epoch})\")\n",
    "        plt.ylabel(\"Loss\")\n",
    "        plt.legend()\n",
    "        plt.savefig(f\"plot_at_epoch_{epoch}\")\n",
    "        self.per_batch_losses = []"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "incident-metallic",
   "metadata": {},
   "source": [
    "让我们试驾一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "irish-bikini",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1563/1563 [==============================] - 18s 11ms/step - loss: 0.2935 - accuracy: 0.9140 - val_loss: 0.1501 - val_accuracy: 0.9580\n",
      "Epoch 2/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1655 - accuracy: 0.9548 - val_loss: 0.1259 - val_accuracy: 0.9659\n",
      "Epoch 3/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1407 - accuracy: 0.9616 - val_loss: 0.1110 - val_accuracy: 0.9712\n",
      "Epoch 4/10\n",
      "1563/1563 [==============================] - 17s 11ms/step - loss: 0.1267 - accuracy: 0.9669 - val_loss: 0.1147 - val_accuracy: 0.9718\n",
      "Epoch 5/10\n",
      "1563/1563 [==============================] - 17s 11ms/step - loss: 0.1211 - accuracy: 0.9698 - val_loss: 0.1105 - val_accuracy: 0.9759\n",
      "Epoch 6/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1114 - accuracy: 0.9727 - val_loss: 0.1130 - val_accuracy: 0.9755\n",
      "Epoch 7/10\n",
      "1563/1563 [==============================] - 17s 11ms/step - loss: 0.1033 - accuracy: 0.9756 - val_loss: 0.1098 - val_accuracy: 0.9767\n",
      "Epoch 8/10\n",
      "1563/1563 [==============================] - 18s 11ms/step - loss: 0.1017 - accuracy: 0.9769 - val_loss: 0.1064 - val_accuracy: 0.9786\n",
      "Epoch 9/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.0948 - accuracy: 0.9771 - val_loss: 0.1117 - val_accuracy: 0.9794\n",
      "Epoch 10/10\n",
      "1563/1563 [==============================] - 16s 11ms/step - loss: 0.0915 - accuracy: 0.9796 - val_loss: 0.1151 - val_accuracy: 0.9786\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fab58fb84f0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg height=\"262.19625pt\" version=\"1.1\" viewBox=\"0 0 390.692561 262.19625\" width=\"390.692561pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2021-07-28T10:57:58.867506</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.4.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 262.19625 \n",
       "L 390.692561 262.19625 \n",
       "L 390.692561 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 43.78125 224.64 \n",
       "L 378.58125 224.64 \n",
       "L 378.58125 7.2 \n",
       "L 43.78125 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"m095d078f0d\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.999432\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(55.818182 239.238437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"97.970448\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 200 -->\n",
       "      <g transform=\"translate(88.426698 239.238437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"136.941464\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 400 -->\n",
       "      <g transform=\"translate(127.397714 239.238437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-34\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"175.91248\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 600 -->\n",
       "      <g transform=\"translate(166.36873 239.238437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" id=\"DejaVuSans-36\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"214.883497\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 800 -->\n",
       "      <g transform=\"translate(205.339747 239.238437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" id=\"DejaVuSans-38\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"253.854513\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 1000 -->\n",
       "      <g transform=\"translate(241.129513 239.238437)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"190.869141\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"292.825529\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 1200 -->\n",
       "      <g transform=\"translate(280.100529 239.238437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"190.869141\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_8\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"331.796545\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 1400 -->\n",
       "      <g transform=\"translate(319.071545 239.238437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"190.869141\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_9\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"370.767561\" xlink:href=\"#m095d078f0d\" y=\"224.64\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1600 -->\n",
       "      <g transform=\"translate(358.042561 239.238437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"190.869141\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- Batch (epoch 9) -->\n",
       "     <g transform=\"translate(171.319531 252.916562)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 1259 2228 \n",
       "L 1259 519 \n",
       "L 2272 519 \n",
       "Q 2781 519 3026 730 \n",
       "Q 3272 941 3272 1375 \n",
       "Q 3272 1813 3026 2020 \n",
       "Q 2781 2228 2272 2228 \n",
       "L 1259 2228 \n",
       "z\n",
       "M 1259 4147 \n",
       "L 1259 2741 \n",
       "L 2194 2741 \n",
       "Q 2656 2741 2882 2914 \n",
       "Q 3109 3088 3109 3444 \n",
       "Q 3109 3797 2882 3972 \n",
       "Q 2656 4147 2194 4147 \n",
       "L 1259 4147 \n",
       "z\n",
       "M 628 4666 \n",
       "L 2241 4666 \n",
       "Q 2963 4666 3353 4366 \n",
       "Q 3744 4066 3744 3513 \n",
       "Q 3744 3084 3544 2831 \n",
       "Q 3344 2578 2956 2516 \n",
       "Q 3422 2416 3680 2098 \n",
       "Q 3938 1781 3938 1306 \n",
       "Q 3938 681 3513 340 \n",
       "Q 3088 0 2303 0 \n",
       "L 628 0 \n",
       "L 628 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-42\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" id=\"DejaVuSans-63\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-68\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1984 4856 \n",
       "Q 1566 4138 1362 3434 \n",
       "Q 1159 2731 1159 2009 \n",
       "Q 1159 1288 1364 580 \n",
       "Q 1569 -128 1984 -844 \n",
       "L 1484 -844 \n",
       "Q 1016 -109 783 600 \n",
       "Q 550 1309 550 2009 \n",
       "Q 550 2706 781 3412 \n",
       "Q 1013 4119 1484 4856 \n",
       "L 1984 4856 \n",
       "z\n",
       "\" id=\"DejaVuSans-28\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" id=\"DejaVuSans-65\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" id=\"DejaVuSans-70\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" id=\"DejaVuSans-6f\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 703 97 \n",
       "L 703 672 \n",
       "Q 941 559 1184 500 \n",
       "Q 1428 441 1663 441 \n",
       "Q 2288 441 2617 861 \n",
       "Q 2947 1281 2994 2138 \n",
       "Q 2813 1869 2534 1725 \n",
       "Q 2256 1581 1919 1581 \n",
       "Q 1219 1581 811 2004 \n",
       "Q 403 2428 403 3163 \n",
       "Q 403 3881 828 4315 \n",
       "Q 1253 4750 1959 4750 \n",
       "Q 2769 4750 3195 4129 \n",
       "Q 3622 3509 3622 2328 \n",
       "Q 3622 1225 3098 567 \n",
       "Q 2575 -91 1691 -91 \n",
       "Q 1453 -91 1209 -44 \n",
       "Q 966 3 703 97 \n",
       "z\n",
       "M 1959 2075 \n",
       "Q 2384 2075 2632 2365 \n",
       "Q 2881 2656 2881 3163 \n",
       "Q 2881 3666 2632 3958 \n",
       "Q 2384 4250 1959 4250 \n",
       "Q 1534 4250 1286 3958 \n",
       "Q 1038 3666 1038 3163 \n",
       "Q 1038 2656 1286 2365 \n",
       "Q 1534 2075 1959 2075 \n",
       "z\n",
       "\" id=\"DejaVuSans-39\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 513 4856 \n",
       "L 1013 4856 \n",
       "Q 1481 4119 1714 3412 \n",
       "Q 1947 2706 1947 2009 \n",
       "Q 1947 1309 1714 600 \n",
       "Q 1481 -109 1013 -844 \n",
       "L 513 -844 \n",
       "Q 928 -128 1133 580 \n",
       "Q 1338 1288 1338 2009 \n",
       "Q 1338 2731 1133 3434 \n",
       "Q 928 4138 513 4856 \n",
       "z\n",
       "\" id=\"DejaVuSans-29\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-42\"/>\n",
       "      <use x=\"68.603516\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"129.882812\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"169.091797\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"224.072266\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"287.451172\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"319.238281\" xlink:href=\"#DejaVuSans-28\"/>\n",
       "      <use x=\"358.251953\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"419.775391\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"483.251953\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"544.433594\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"599.414062\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"662.792969\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"694.580078\" xlink:href=\"#DejaVuSans-39\"/>\n",
       "      <use x=\"758.203125\" xlink:href=\"#DejaVuSans-29\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m12cb7fea7e\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m12cb7fea7e\" y=\"201.894624\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.1 -->\n",
       "      <g transform=\"translate(20.878125 205.693843)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" id=\"DejaVuSans-2e\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m12cb7fea7e\" y=\"159.572008\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.2 -->\n",
       "      <g transform=\"translate(20.878125 163.371227)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m12cb7fea7e\" y=\"117.249392\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 0.3 -->\n",
       "      <g transform=\"translate(20.878125 121.048611)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" id=\"DejaVuSans-33\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m12cb7fea7e\" y=\"74.926776\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 0.4 -->\n",
       "      <g transform=\"translate(20.878125 78.725994)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m12cb7fea7e\" y=\"32.604159\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_15\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(20.878125 36.403378)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-35\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_16\">\n",
       "     <!-- Loss -->\n",
       "     <g transform=\"translate(14.798438 126.887187)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 628 4666 \n",
       "L 1259 4666 \n",
       "L 1259 531 \n",
       "L 3531 531 \n",
       "L 3531 0 \n",
       "L 628 0 \n",
       "L 628 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-4c\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" id=\"DejaVuSans-73\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-4c\"/>\n",
       "      <use x=\"53.962891\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"115.144531\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"167.244141\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_15\">\n",
       "    <path clip-path=\"url(#pe8d789db62)\" d=\"M 58.999432 126.809238 \n",
       "L 59.194287 17.083636 \n",
       "L 59.583997 125.987254 \n",
       "L 59.973707 158.235678 \n",
       "L 60.363417 179.168763 \n",
       "L 61.142838 197.228544 \n",
       "L 61.727403 204.783004 \n",
       "L 61.922258 200.735181 \n",
       "L 62.311968 205.332258 \n",
       "L 62.506823 197.783758 \n",
       "L 63.091389 203.90233 \n",
       "L 64.065664 210.068861 \n",
       "L 64.650229 212.757244 \n",
       "L 64.845084 212.902361 \n",
       "L 65.039939 212.043 \n",
       "L 65.234794 205.314284 \n",
       "L 65.429649 206.440034 \n",
       "L 65.624505 205.173235 \n",
       "L 66.403925 208.734856 \n",
       "L 66.59878 207.647523 \n",
       "L 66.98849 209.381345 \n",
       "L 67.183345 209.668741 \n",
       "L 67.3782 210.452236 \n",
       "L 67.573055 208.946614 \n",
       "L 67.76791 209.380758 \n",
       "L 67.962766 210.095244 \n",
       "L 68.157621 209.213533 \n",
       "L 68.547331 210.266117 \n",
       "L 68.742186 206.081793 \n",
       "L 68.937041 206.771721 \n",
       "L 69.131896 206.937707 \n",
       "L 69.521606 208.250001 \n",
       "L 69.716461 208.246662 \n",
       "L 69.911316 207.652852 \n",
       "L 70.301027 208.54891 \n",
       "L 71.080447 210.627184 \n",
       "L 72.444432 213.235243 \n",
       "L 72.639287 212.538974 \n",
       "L 72.834143 212.754831 \n",
       "L 73.028998 213.176832 \n",
       "L 73.223853 212.853797 \n",
       "L 73.418708 213.055866 \n",
       "L 73.613563 213.447219 \n",
       "L 73.808418 213.381858 \n",
       "L 74.003273 213.770342 \n",
       "L 74.198128 213.624352 \n",
       "L 74.392983 213.810767 \n",
       "L 74.587838 213.794288 \n",
       "L 74.782693 214.161251 \n",
       "L 74.977548 214.075069 \n",
       "L 75.172404 214.250858 \n",
       "L 75.367259 213.858426 \n",
       "L 75.562114 212.29326 \n",
       "L 75.756969 212.36705 \n",
       "L 75.951824 212.714394 \n",
       "L 76.146679 212.730986 \n",
       "L 77.120954 214.362399 \n",
       "L 77.315809 214.317868 \n",
       "L 77.510665 214.452889 \n",
       "L 77.70552 214.756364 \n",
       "L 77.900375 213.341808 \n",
       "L 78.09523 213.64282 \n",
       "L 78.290085 213.658003 \n",
       "L 78.679795 214.188912 \n",
       "L 78.87465 211.721949 \n",
       "L 79.26436 211.277015 \n",
       "L 79.459215 211.579566 \n",
       "L 79.65407 211.297369 \n",
       "L 79.848925 210.621511 \n",
       "L 80.823201 211.76543 \n",
       "L 81.018056 211.815043 \n",
       "L 81.212911 212.09373 \n",
       "L 81.407766 212.018821 \n",
       "L 81.602621 212.26496 \n",
       "L 81.992331 212.305486 \n",
       "L 82.187186 210.134178 \n",
       "L 83.356317 211.592592 \n",
       "L 83.551172 211.143745 \n",
       "L 83.940882 210.948008 \n",
       "L 84.330592 210.266839 \n",
       "L 84.720302 210.707141 \n",
       "L 84.915158 210.692125 \n",
       "L 85.110013 210.287541 \n",
       "L 85.499723 208.251489 \n",
       "L 87.253419 209.938394 \n",
       "L 87.448274 209.733497 \n",
       "L 87.643129 209.290007 \n",
       "L 88.812259 210.286812 \n",
       "L 89.007114 209.749515 \n",
       "L 89.201969 209.965928 \n",
       "L 89.396824 209.627193 \n",
       "L 89.59168 209.845258 \n",
       "L 89.786535 209.105278 \n",
       "L 89.98139 209.14348 \n",
       "L 90.3711 209.492302 \n",
       "L 90.565955 209.413451 \n",
       "L 90.76081 209.603749 \n",
       "L 91.15052 209.20833 \n",
       "L 92.709361 210.587563 \n",
       "L 92.904216 210.48807 \n",
       "L 93.293926 210.819584 \n",
       "L 93.488781 210.65924 \n",
       "L 94.073346 210.674489 \n",
       "L 94.463057 210.942789 \n",
       "L 94.657912 210.203818 \n",
       "L 95.242477 210.64739 \n",
       "L 96.021897 211.17832 \n",
       "L 96.216752 210.548254 \n",
       "L 96.606462 210.876592 \n",
       "L 96.801318 210.866401 \n",
       "L 97.191028 210.357234 \n",
       "L 97.580738 210.655696 \n",
       "L 97.775593 210.604795 \n",
       "L 98.749868 211.048579 \n",
       "L 98.944723 210.681275 \n",
       "L 99.139578 210.766429 \n",
       "L 99.724144 210.58528 \n",
       "L 99.918999 209.8997 \n",
       "L 100.113854 209.812382 \n",
       "L 100.893274 210.336498 \n",
       "L 101.088129 210.360148 \n",
       "L 102.452115 211.353066 \n",
       "L 102.841825 211.438425 \n",
       "L 103.231535 211.199485 \n",
       "L 103.42639 211.276409 \n",
       "L 103.621245 211.131346 \n",
       "L 104.010956 211.311922 \n",
       "L 104.205811 210.956052 \n",
       "L 104.985231 211.358975 \n",
       "L 105.180086 211.250139 \n",
       "L 105.959506 211.636978 \n",
       "L 106.154361 211.542751 \n",
       "L 106.349216 211.625295 \n",
       "L 106.544072 211.001664 \n",
       "L 107.128637 211.152864 \n",
       "L 107.323492 211.239447 \n",
       "L 107.518347 211.090388 \n",
       "L 108.297767 211.337911 \n",
       "L 108.492622 211.465464 \n",
       "L 108.687477 211.218008 \n",
       "L 108.882333 211.327234 \n",
       "L 109.077188 211.01137 \n",
       "L 109.466898 211.142947 \n",
       "L 109.856608 211.371225 \n",
       "L 110.051463 210.498158 \n",
       "L 110.246318 210.621993 \n",
       "L 110.636028 210.697606 \n",
       "L 110.830883 210.220565 \n",
       "L 111.415449 210.561479 \n",
       "L 111.805159 210.400759 \n",
       "L 112.194869 210.606949 \n",
       "L 112.389724 210.466937 \n",
       "L 112.584579 210.526875 \n",
       "L 112.779434 209.852634 \n",
       "L 113.169144 210.00016 \n",
       "L 113.363999 210.12176 \n",
       "L 113.75371 209.963245 \n",
       "L 114.338275 210.11272 \n",
       "L 115.507405 210.694175 \n",
       "L 115.897115 210.906312 \n",
       "L 116.091971 210.803938 \n",
       "L 117.066246 211.232368 \n",
       "L 117.261101 211.151902 \n",
       "L 119.404507 211.941364 \n",
       "L 119.794217 211.538699 \n",
       "L 120.183927 211.661444 \n",
       "L 120.378782 211.560075 \n",
       "L 120.768492 211.684204 \n",
       "L 120.963348 211.634896 \n",
       "L 121.158203 211.708772 \n",
       "L 121.353058 211.046239 \n",
       "L 121.547913 210.916853 \n",
       "L 122.327333 211.241049 \n",
       "L 122.911898 211.158341 \n",
       "L 124.860449 212.059394 \n",
       "L 125.055304 211.898744 \n",
       "L 125.250159 211.992816 \n",
       "L 125.834725 211.779695 \n",
       "L 126.02958 211.828498 \n",
       "L 126.224435 211.600097 \n",
       "L 126.41929 211.684636 \n",
       "L 126.614145 211.523627 \n",
       "L 127.19871 211.71225 \n",
       "L 127.393565 211.646081 \n",
       "L 127.58842 211.726465 \n",
       "L 127.97813 211.55235 \n",
       "L 128.172986 210.566757 \n",
       "L 128.367841 210.628631 \n",
       "L 128.562696 210.439526 \n",
       "L 128.757551 210.532264 \n",
       "L 128.952406 210.436678 \n",
       "L 130.511247 210.960331 \n",
       "L 131.290667 211.216939 \n",
       "L 131.875232 211.162712 \n",
       "L 132.070087 211.242297 \n",
       "L 132.264942 211.176 \n",
       "L 132.654652 211.251647 \n",
       "L 132.849507 211.221965 \n",
       "L 133.044363 211.30647 \n",
       "L 133.434073 211.054964 \n",
       "L 134.408348 211.268189 \n",
       "L 134.798058 211.392346 \n",
       "L 134.992913 211.330044 \n",
       "L 135.577479 211.503792 \n",
       "L 135.772334 211.437066 \n",
       "L 136.551754 211.735855 \n",
       "L 136.746609 211.161791 \n",
       "L 137.331174 211.324115 \n",
       "L 137.526029 211.003152 \n",
       "L 138.890015 211.05469 \n",
       "L 139.08487 210.89987 \n",
       "L 139.279725 210.442934 \n",
       "L 139.47458 210.464925 \n",
       "L 139.669435 210.342521 \n",
       "L 140.254001 210.547626 \n",
       "L 140.448856 210.07541 \n",
       "L 140.838566 210.12637 \n",
       "L 141.033421 210.026685 \n",
       "L 141.423131 210.087238 \n",
       "L 141.617986 209.53835 \n",
       "L 142.007696 209.696496 \n",
       "L 142.202551 209.631422 \n",
       "L 142.787117 209.834941 \n",
       "L 142.981972 209.685242 \n",
       "L 143.176827 209.744773 \n",
       "L 143.371682 209.274644 \n",
       "L 143.566537 209.218487 \n",
       "L 143.761392 208.823189 \n",
       "L 145.320233 209.399561 \n",
       "L 147.268783 209.575445 \n",
       "L 147.463639 209.651347 \n",
       "L 147.658494 209.176495 \n",
       "L 148.048204 209.179431 \n",
       "L 148.437914 209.315583 \n",
       "L 148.632769 209.127909 \n",
       "L 149.607044 209.437372 \n",
       "L 149.8019 209.317144 \n",
       "L 149.996755 209.390303 \n",
       "L 150.19161 209.153634 \n",
       "L 150.386465 209.219017 \n",
       "L 150.58132 209.153561 \n",
       "L 150.776175 208.829486 \n",
       "L 152.919581 209.504159 \n",
       "L 153.893856 209.68651 \n",
       "L 154.868132 209.9868 \n",
       "L 155.062987 209.885251 \n",
       "L 156.426972 210.14611 \n",
       "L 156.621827 209.877072 \n",
       "L 157.206393 210.051946 \n",
       "L 157.401248 209.917762 \n",
       "L 157.985813 210.102721 \n",
       "L 158.375523 209.577942 \n",
       "L 158.765233 209.711414 \n",
       "L 158.960088 209.535354 \n",
       "L 159.544654 209.729681 \n",
       "L 159.739509 209.484305 \n",
       "L 159.934364 209.547103 \n",
       "L 160.129219 209.427436 \n",
       "L 161.493204 209.736637 \n",
       "L 162.46748 209.981285 \n",
       "L 162.662335 209.708532 \n",
       "L 163.2469 209.844927 \n",
       "L 163.441755 209.773307 \n",
       "L 163.63661 209.477349 \n",
       "L 163.831465 209.539472 \n",
       "L 164.221176 209.435171 \n",
       "L 164.805741 209.548765 \n",
       "L 165.195451 209.577797 \n",
       "L 165.585161 209.670765 \n",
       "L 166.364581 209.427323 \n",
       "L 166.559436 209.490085 \n",
       "L 166.754292 209.098735 \n",
       "L 167.728567 209.301749 \n",
       "L 168.118277 209.425257 \n",
       "L 168.507987 209.352095 \n",
       "L 168.897697 209.220716 \n",
       "L 169.287408 209.341837 \n",
       "L 169.482263 209.259514 \n",
       "L 170.261683 209.329571 \n",
       "L 170.651393 209.328565 \n",
       "L 170.846248 209.038996 \n",
       "L 171.235958 209.031989 \n",
       "L 171.430813 208.695553 \n",
       "L 172.015379 208.778182 \n",
       "L 172.210234 208.608293 \n",
       "L 172.599944 208.621811 \n",
       "L 173.769074 208.752949 \n",
       "L 175.13306 209.062135 \n",
       "L 175.52277 209.082659 \n",
       "L 176.107335 209.223365 \n",
       "L 176.302191 208.999069 \n",
       "L 179.809582 209.303682 \n",
       "L 180.004437 209.02661 \n",
       "L 180.589002 209.156279 \n",
       "L 180.783857 208.964266 \n",
       "L 182.732408 209.404493 \n",
       "L 183.316973 209.349979 \n",
       "L 184.680959 209.634004 \n",
       "L 184.875814 209.316898 \n",
       "L 186.62951 209.404849 \n",
       "L 187.01922 209.406372 \n",
       "L 187.993495 209.563737 \n",
       "L 188.772916 209.69216 \n",
       "L 189.162626 209.513133 \n",
       "L 189.552336 209.609503 \n",
       "L 189.747191 209.468943 \n",
       "L 191.695742 209.640547 \n",
       "L 193.059727 209.903916 \n",
       "L 193.449438 209.891965 \n",
       "L 194.228858 209.999091 \n",
       "L 194.618568 209.811367 \n",
       "L 195.787699 210.053094 \n",
       "L 196.372264 210.056345 \n",
       "L 196.761974 210.092706 \n",
       "L 196.956829 209.939705 \n",
       "L 197.346539 210.034941 \n",
       "L 197.736249 209.53174 \n",
       "L 198.12596 209.59351 \n",
       "L 198.51567 209.592372 \n",
       "L 198.90538 209.573367 \n",
       "L 199.489945 209.711701 \n",
       "L 199.879655 209.554974 \n",
       "L 200.464221 209.634272 \n",
       "L 200.659076 209.547343 \n",
       "L 201.048786 209.579326 \n",
       "L 201.438496 209.466426 \n",
       "L 201.633351 209.467079 \n",
       "L 201.828206 209.356358 \n",
       "L 202.412771 209.441755 \n",
       "L 202.802482 209.401708 \n",
       "L 202.997337 209.447336 \n",
       "L 203.387047 209.295336 \n",
       "L 203.581902 209.34202 \n",
       "L 203.776757 209.152814 \n",
       "L 204.556177 209.209765 \n",
       "L 205.140742 209.279118 \n",
       "L 205.530453 209.1642 \n",
       "L 206.115018 209.254684 \n",
       "L 207.284148 208.812001 \n",
       "L 208.648134 209.00945 \n",
       "L 209.037844 208.956115 \n",
       "L 210.40183 208.99852 \n",
       "L 211.18125 209.104852 \n",
       "L 212.35038 209.135389 \n",
       "L 212.545236 209.050483 \n",
       "L 213.129801 209.084914 \n",
       "L 213.714366 209.151445 \n",
       "L 213.909221 208.868312 \n",
       "L 215.273207 208.874786 \n",
       "L 217.026902 209.10903 \n",
       "L 218.585743 209.337473 \n",
       "L 219.170308 209.152729 \n",
       "L 219.365163 208.916024 \n",
       "L 221.703424 209.214637 \n",
       "L 225.210816 209.431873 \n",
       "L 226.964512 209.502191 \n",
       "L 227.743932 209.128161 \n",
       "L 229.887338 209.429612 \n",
       "L 230.277048 209.451417 \n",
       "L 230.471903 208.783051 \n",
       "L 230.861613 208.682546 \n",
       "L 231.446178 208.685091 \n",
       "L 232.420454 208.637249 \n",
       "L 233.005019 208.666218 \n",
       "L 236.317555 208.962668 \n",
       "L 236.707266 208.838996 \n",
       "L 237.096976 208.840989 \n",
       "L 237.681541 208.62148 \n",
       "L 240.214657 208.810273 \n",
       "L 240.604367 208.727818 \n",
       "L 241.383788 208.8712 \n",
       "L 241.773498 208.641777 \n",
       "L 242.747773 208.603928 \n",
       "L 243.137483 208.384365 \n",
       "L 243.527193 208.344836 \n",
       "L 244.891179 208.403228 \n",
       "L 245.280889 208.433645 \n",
       "L 245.475744 208.125986 \n",
       "L 245.865454 208.198789 \n",
       "L 246.255165 208.101891 \n",
       "L 248.203715 208.244852 \n",
       "L 248.39857 208.112644 \n",
       "L 248.788281 208.183385 \n",
       "L 248.983136 208.023807 \n",
       "L 249.762556 208.043638 \n",
       "L 250.152266 208.057399 \n",
       "L 250.931687 208.125803 \n",
       "L 251.321397 207.974344 \n",
       "L 251.516252 208.010705 \n",
       "L 251.711107 207.866263 \n",
       "L 251.905962 207.900154 \n",
       "L 252.100817 207.807735 \n",
       "L 252.880237 207.859448 \n",
       "L 253.269947 207.838927 \n",
       "L 253.854513 207.766436 \n",
       "L 254.439078 207.765506 \n",
       "L 256.972194 207.824664 \n",
       "L 257.167049 207.398669 \n",
       "L 258.33618 207.487446 \n",
       "L 258.920745 207.239725 \n",
       "L 262.817846 207.637877 \n",
       "L 263.012702 207.46258 \n",
       "L 263.207557 207.497531 \n",
       "L 263.402412 206.903531 \n",
       "L 263.597267 206.751852 \n",
       "L 264.181832 206.781471 \n",
       "L 264.571542 206.746072 \n",
       "L 265.156107 206.795796 \n",
       "L 266.520093 206.904616 \n",
       "L 266.909803 206.895522 \n",
       "L 267.299513 206.891142 \n",
       "L 268.468644 206.965156 \n",
       "L 268.858354 206.918333 \n",
       "L 270.027484 207.041954 \n",
       "L 271.196615 207.145675 \n",
       "L 271.586325 206.997953 \n",
       "L 273.145166 207.130993 \n",
       "L 273.534876 206.895128 \n",
       "L 273.924586 206.796228 \n",
       "L 274.509151 206.879472 \n",
       "L 274.898861 206.523842 \n",
       "L 275.873137 206.559111 \n",
       "L 276.067992 206.590984 \n",
       "L 276.457702 206.541178 \n",
       "L 278.990818 206.736338 \n",
       "L 279.575383 206.597042 \n",
       "L 280.549659 206.666118 \n",
       "L 280.939369 206.499196 \n",
       "L 281.134224 206.53199 \n",
       "L 281.523934 206.325507 \n",
       "L 282.693065 206.465247 \n",
       "L 283.472485 206.329666 \n",
       "L 285.421036 206.413732 \n",
       "L 286.395311 206.461571 \n",
       "L 288.928427 206.719181 \n",
       "L 289.512993 206.737183 \n",
       "L 289.902703 206.391877 \n",
       "L 290.876978 206.269224 \n",
       "L 291.851253 206.398776 \n",
       "L 292.046109 206.288213 \n",
       "L 293.799804 206.477488 \n",
       "L 294.38437 206.485356 \n",
       "L 296.722631 206.687276 \n",
       "L 297.891761 206.764393 \n",
       "L 299.645457 206.986734 \n",
       "L 300.035167 206.895314 \n",
       "L 300.814587 206.926203 \n",
       "L 301.204297 206.689991 \n",
       "L 302.568283 206.801911 \n",
       "L 303.152848 206.748081 \n",
       "L 303.932269 206.737306 \n",
       "L 305.101399 206.581815 \n",
       "L 305.491109 206.369545 \n",
       "L 307.04995 206.520376 \n",
       "L 307.634515 206.557727 \n",
       "L 308.803646 206.652215 \n",
       "L 309.193356 206.565834 \n",
       "L 309.388211 206.59504 \n",
       "L 309.583066 206.490846 \n",
       "L 311.726472 206.758411 \n",
       "L 312.116182 206.62638 \n",
       "L 313.675023 206.717519 \n",
       "L 314.064733 206.558581 \n",
       "L 314.649298 206.612692 \n",
       "L 315.039008 206.581528 \n",
       "L 315.818428 206.603427 \n",
       "L 319.130965 206.477599 \n",
       "L 319.32582 206.230943 \n",
       "L 319.520675 206.25667 \n",
       "L 319.71553 205.961836 \n",
       "L 320.300095 205.937871 \n",
       "L 327.314878 206.062018 \n",
       "L 329.263429 206.229577 \n",
       "L 330.237704 206.313061 \n",
       "L 330.43256 206.114817 \n",
       "L 332.38111 206.184312 \n",
       "L 332.77082 206.152641 \n",
       "L 333.160531 206.085933 \n",
       "L 334.134806 206.170898 \n",
       "L 335.109081 206.222025 \n",
       "L 335.693647 206.208116 \n",
       "L 337.057632 206.311705 \n",
       "L 337.447342 206.159001 \n",
       "L 338.226763 206.165689 \n",
       "L 340.565024 206.054983 \n",
       "L 340.954734 206.012152 \n",
       "L 341.344444 206.025346 \n",
       "L 341.734154 205.997196 \n",
       "L 343.682705 206.150443 \n",
       "L 345.436401 206.026544 \n",
       "L 346.020966 206.058644 \n",
       "L 346.410676 206.028086 \n",
       "L 348.748937 205.889679 \n",
       "L 348.943792 205.915148 \n",
       "L 349.333502 205.783152 \n",
       "L 349.918068 205.766584 \n",
       "L 351.282053 205.735222 \n",
       "L 351.476908 205.760413 \n",
       "L 351.866618 205.51162 \n",
       "L 354.399734 205.529855 \n",
       "L 354.59459 205.552899 \n",
       "L 354.9843 205.486475 \n",
       "L 356.54314 205.617119 \n",
       "L 356.737995 205.428159 \n",
       "L 358.881401 205.313899 \n",
       "L 360.440242 205.472806 \n",
       "L 360.635097 205.489288 \n",
       "L 360.829952 205.373102 \n",
       "L 362.193938 205.418784 \n",
       "L 363.363068 205.480383 \n",
       "L 363.363068 205.480383 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 43.78125 224.64 \n",
       "L 43.78125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 378.58125 224.64 \n",
       "L 378.58125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 43.78125 224.64 \n",
       "L 378.58125 224.64 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 43.78125 7.2 \n",
       "L 378.58125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 202.001563 29.878125 \n",
       "L 371.58125 29.878125 \n",
       "Q 373.58125 29.878125 373.58125 27.878125 \n",
       "L 373.58125 14.2 \n",
       "Q 373.58125 12.2 371.58125 12.2 \n",
       "L 202.001563 12.2 \n",
       "Q 200.001563 12.2 200.001563 14.2 \n",
       "L 200.001563 27.878125 \n",
       "Q 200.001563 29.878125 202.001563 29.878125 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_16\">\n",
       "     <path d=\"M 204.001563 20.298437 \n",
       "L 224.001563 20.298437 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_17\"/>\n",
       "    <g id=\"text_17\">\n",
       "     <!-- Training loss for each batch -->\n",
       "     <g transform=\"translate(232.001563 23.798437)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M -19 4666 \n",
       "L 3928 4666 \n",
       "L 3928 4134 \n",
       "L 2272 4134 \n",
       "L 2272 0 \n",
       "L 1638 0 \n",
       "L 1638 4134 \n",
       "L -19 4134 \n",
       "L -19 4666 \n",
       "z\n",
       "\" id=\"DejaVuSans-54\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" id=\"DejaVuSans-72\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2906 1791 \n",
       "Q 2906 2416 2648 2759 \n",
       "Q 2391 3103 1925 3103 \n",
       "Q 1463 3103 1205 2759 \n",
       "Q 947 2416 947 1791 \n",
       "Q 947 1169 1205 825 \n",
       "Q 1463 481 1925 481 \n",
       "Q 2391 481 2648 825 \n",
       "Q 2906 1169 2906 1791 \n",
       "z\n",
       "M 3481 434 \n",
       "Q 3481 -459 3084 -895 \n",
       "Q 2688 -1331 1869 -1331 \n",
       "Q 1566 -1331 1297 -1286 \n",
       "Q 1028 -1241 775 -1147 \n",
       "L 775 -588 \n",
       "Q 1028 -725 1275 -790 \n",
       "Q 1522 -856 1778 -856 \n",
       "Q 2344 -856 2625 -561 \n",
       "Q 2906 -266 2906 331 \n",
       "L 2906 616 \n",
       "Q 2728 306 2450 153 \n",
       "Q 2172 0 1784 0 \n",
       "Q 1141 0 747 490 \n",
       "Q 353 981 353 1791 \n",
       "Q 353 2603 747 3093 \n",
       "Q 1141 3584 1784 3584 \n",
       "Q 2172 3584 2450 3431 \n",
       "Q 2728 3278 2906 2969 \n",
       "L 2906 3500 \n",
       "L 3481 3500 \n",
       "L 3481 434 \n",
       "z\n",
       "\" id=\"DejaVuSans-67\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" id=\"DejaVuSans-66\" transform=\"scale(0.015625)\"/>\n",
       "       <path d=\"M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "M 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2969 \n",
       "z\n",
       "\" id=\"DejaVuSans-62\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-54\"/>\n",
       "      <use x=\"46.333984\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"87.447266\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"148.726562\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"176.509766\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"239.888672\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"267.671875\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"331.050781\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"394.527344\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"426.314453\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"454.097656\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"515.279297\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"567.378906\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"619.478516\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"651.265625\" xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use x=\"686.470703\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"747.652344\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"788.765625\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"820.552734\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"882.076172\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"943.355469\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"998.335938\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"1061.714844\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "      <use x=\"1093.501953\" xlink:href=\"#DejaVuSans-62\"/>\n",
       "      <use x=\"1156.978516\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"1218.257812\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"1257.466797\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"1312.447266\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pe8d789db62\">\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"43.78125\" y=\"7.2\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2min 51s (started: 2021-07-28 10:55:07 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = get_mnist_model()\n",
    "\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"])\n",
    "\n",
    "model.fit(train_images, \n",
    "          train_labels,\n",
    "          epochs=10,\n",
    "          callbacks=[LossHistory()],\n",
    "          validation_data=(val_images, val_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "informed-winter",
   "metadata": {},
   "source": [
    "### 使用 TensorBoard 进行监控和可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "complicated-driving",
   "metadata": {},
   "source": [
    "为了进行良好的研究或开发良好的模型，您需要对实验期间模型内部发生的情况进行丰富、频繁的反馈。 这就是运行实验的重点：获取有关模型执行情况的信息——尽可能多的信息。 取得进展是一个反复的过程——一个循环：你从一个想法开始，并将其表达为一个实验，试图验证或否定你的想法。 您运行此实验并处理它生成的信息。 这激发了你的下一个想法。 您能够运行的此循环的迭代次数越多，您的想法就越精致和强大。 Keras 可以帮助您在尽可能短的时间内从想法到实验，而快速 GPU 可以帮助您尽快从实验到结果。 但是如何处理实验结果呢？ 这就是 TensorBoard 的用武之地。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "returning-welding",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/008i3skNgy1gswm6ymyvjj31b60tggo9.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "earned-scheme",
   "metadata": {},
   "source": [
    "TensorBoard 是一个基于浏览器的应用程序，您可以在本地运行。 这是在训练期间监控模型内部发生的一切的最佳方式。 使用 TensorBoard，您可以：\n",
    "* 在训练期间可视化监控指标\n",
    "* 可视化您的模型架构\n",
    "* 可视化激活和梯度的直方图\n",
    "* 探索 3D 嵌入"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dress-vegetarian",
   "metadata": {},
   "source": [
    "如果您监控的信息不仅仅是模型的最终损失，您可以对模型做什么和不做什么有更清晰的认识，并且可以更快地取得进展。\n",
    "\n",
    "将 TensorBoard 与 Keras 模型和 fit 方法一起使用的最简单方法是 keras.callbacks.TensorBoard 回调。\n",
    "\n",
    "在最简单的情况下，只需指定您希望回调写入日志的位置，就可以了："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "polyphonic-corner",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.2940 - accuracy: 0.9117 - val_loss: 0.1513 - val_accuracy: 0.9584\n",
      "Epoch 2/10\n",
      "1563/1563 [==============================] - 17s 11ms/step - loss: 0.1654 - accuracy: 0.9534 - val_loss: 0.1241 - val_accuracy: 0.9682\n",
      "Epoch 3/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1403 - accuracy: 0.9621 - val_loss: 0.1125 - val_accuracy: 0.9714\n",
      "Epoch 4/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1283 - accuracy: 0.9663 - val_loss: 0.1054 - val_accuracy: 0.9743\n",
      "Epoch 5/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1175 - accuracy: 0.9699 - val_loss: 0.1067 - val_accuracy: 0.9756\n",
      "Epoch 6/10\n",
      "1563/1563 [==============================] - 15s 10ms/step - loss: 0.1119 - accuracy: 0.9728 - val_loss: 0.1043 - val_accuracy: 0.9768\n",
      "Epoch 7/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.1066 - accuracy: 0.9748 - val_loss: 0.1100 - val_accuracy: 0.9753\n",
      "Epoch 8/10\n",
      "1563/1563 [==============================] - 15s 10ms/step - loss: 0.1016 - accuracy: 0.9755 - val_loss: 0.1153 - val_accuracy: 0.9786\n",
      "Epoch 9/10\n",
      "1563/1563 [==============================] - 17s 11ms/step - loss: 0.1005 - accuracy: 0.9765 - val_loss: 0.1096 - val_accuracy: 0.9783\n",
      "Epoch 10/10\n",
      "1563/1563 [==============================] - 16s 10ms/step - loss: 0.0964 - accuracy: 0.9777 - val_loss: 0.1173 - val_accuracy: 0.9764\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fe55c0a2820>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2min 40s (started: 2021-07-28 14:00:21 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = get_mnist_model()\n",
    "\n",
    "model.compile(optimizer=\"rmsprop\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"accuracy\"])\n",
    "\n",
    "tensorboard = keras.callbacks.TensorBoard(log_dir=\"./tensorboard_log\",)\n",
    "\n",
    "model.fit(train_images, train_labels,\n",
    "          epochs=10,\n",
    "          validation_data=(val_images, val_labels),\n",
    "          callbacks=[tensorboard])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "responsible-burlington",
   "metadata": {},
   "source": [
    "一旦模型开始运行，它将在目标位置写入日志。 如果您在本地机器上运行 Python 脚本，则可以使用以下命令启动本地 TensorBoard 服务器（请注意，如果您通过 TensorFlow 安装了 tensorboard 可执行文件，则应该已经可用；如果没有，您可以通过手动安装 TensorBoard pip pip 安装张量板）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "loaded-coating",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash: tensorboard: 未找到命令\n"
     ]
    }
   ],
   "source": [
    "!tensorboard --logdir ./tensorboard_log"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "leading-margin",
   "metadata": {},
   "source": [
    "然后，您可以导航到命令返回的 URL 以访问 TensorBoard 界面。\n",
    "\n",
    "如果您在 Colab 笔记本中运行脚本，则可以使用以下命令将嵌入式 TensorBoard 实例作为笔记本的一部分运行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "attended-music",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ERROR: Failed to launch TensorBoard (exited with 1).\n",
       "Contents of stderr:\n",
       "2021-07-28 14:40:14.777058: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0\n",
       "/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard_data_server/bin/server: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard_data_server/bin/server)\n",
       "Traceback (most recent call last):\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/bin/tensorboard\", line 8, in <module>\n",
       "    sys.exit(run_main())\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard/main.py\", line 46, in run_main\n",
       "    app.run(tensorboard.main, flags_parser=tensorboard.configure)\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/absl/app.py\", line 312, in run\n",
       "    _run_main(main, args)\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/absl/app.py\", line 258, in _run_main\n",
       "    sys.exit(main(argv))\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard/program.py\", line 276, in main\n",
       "    return runner(self.flags) or 0\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard/program.py\", line 292, in _run_serve_subcommand\n",
       "    server = self._make_server()\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard/program.py\", line 467, in _make_server\n",
       "    app = application.TensorBoardWSGIApp(\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard/backend/application.py\", line 139, in TensorBoardWSGIApp\n",
       "    return TensorBoardWSGI(\n",
       "  File \"/public/huangwei/miniconda3/envs/tensorflow/lib/python3.8/site-packages/tensorboard/backend/application.py\", line 252, in __init__\n",
       "    raise ValueError(\n",
       "ValueError: Duplicate plugins for name projector"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%load_ext tensorboard\n",
    "%tensorboard --logdir ./tensorboard_log"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fitted-benefit",
   "metadata": {},
   "source": [
    "在 TensorBoard 界面中，您将能够监控训练和评估指标的实时图表："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "grave-graduate",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/008i3skNgy1gswnrn8eb8j316a0u0why.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "athletic-apollo",
   "metadata": {},
   "source": [
    "## 编写自己的训练和评估循环"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "human-shade",
   "metadata": {},
   "source": [
    "fit() 工作流在易用性和灵活性之间取得了很好的平衡。 这是你大部分时间都会用到的。 然而，它并不意味着支持深度学习研究人员可能想做的所有事情——即使是自定义指标、自定义损失和自定义回调。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fixed-humanity",
   "metadata": {},
   "source": [
    "毕竟，内置的 fit() 工作流程仅专注于监督学习：一种设置，其中存在与输入数据相关的已知（也称为标签或注释），以及根据这些目标计算损失的目标 以及模型的预测。 然而，并非所有形式的机器学习都属于这一类。 还有其他不存在明确目标的设置，例如生成学习（我们将在第 12 章中介绍）、自监督学习（从输入中获得目标）或强化学习（学习由偶然驱动） 奖励”——很像训练狗）。 即使你正在做常规的监督学习，作为一名研究人员，你可能想要添加一些需要低级灵活性的新颖花里胡哨"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "minute-plane",
   "metadata": {},
   "source": [
    "每当您发现自己处于内置 fit() 不够用的情况时，您将需要编写自己的自定义训练逻辑。 您已经在第 2 章和第 3 章中看到了低级训练循环的简单示例。提醒一下，典型训练循环的内容如下所示：\n",
    "* 在“梯度磁带”中运行“前向传递”（计算模型的输出）以获得当前批次数据的损失值。\n",
    "* 检索与模型权重有关的损失梯度。\n",
    "* 更新模型的权重以降低当前批次数据的损失值。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "standard-netherlands",
   "metadata": {},
   "source": [
    "根据需要对尽可能多的批次重复这些步骤。 这本质上是 fit() 引擎盖下的内容。 在本节中，您将学习从头开始重新实现，这将 fit() 为您提供编写您可能想出的任何训练算法所需的所有知识。 让我们来看看细节。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "negative-avenue",
   "metadata": {},
   "source": [
    "根据需要对尽可能多的批次重复这些步骤。 这本质上是 fit() 在引擎盖下所做的。 在本节中，您将学习从头开始重新实现 fit()，这将为您提供编写您可能想出的任何训练算法所需的所有知识。 \n",
    "\n",
    "让我们来看看细节。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "iraqi-accident",
   "metadata": {},
   "source": [
    "### 训练与推理"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "three-corporation",
   "metadata": {},
   "source": [
    "在您目前看到的低级训练循环示例中，第 1 步（前向传递）是通过`predictions = model(inputs)`完成的，第 2 步（检索由梯度磁带计算的梯度）是通过`gradients = tape.gradient(loss, model.weights)` 完成的。 在一般情况下，您实际上需要考虑两个微妙之处。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acute-hypothesis",
   "metadata": {},
   "source": [
    "一些 Keras 层，例如层，在训练期间和 Dropout 训练期间（当您使用它们生成预测时）具有不同的行为。 这些层在其方法中公开了推理训练布尔参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "intelligent-tradition",
   "metadata": {},
   "source": [
    "一些 Keras 层，例如 Dropout 层，在推理期间和训练期间（当您使用它们生成预测时）具有不同的行为。 这些层在它们的 call() 方法中公开了一个训练布尔参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "continued-exploration",
   "metadata": {},
   "source": [
    "调用 dropout(inputs, training=True) 将删除一些激活条目，而调用 dropout(inputs, training=False) 什么也不做。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fiscal-powell",
   "metadata": {},
   "source": [
    "通过扩展，Functional 模型和 Sequential 模型也在它们的 call() 方法中公开了这个训练参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "comfortable-accused",
   "metadata": {},
   "source": [
    "在前向传递期间调用 Keras 模型时，请记住传递 training=True！ 因此，我们的前向传递变成了predictions = model(inputs, training=True)。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suffering-richards",
   "metadata": {},
   "source": [
    "此外，请注意，当您检索模型权重的梯度时，不应使用 tape.gradients(loss, model.weights)，而应使用 tape.gradients(loss, model.trainable_weights)。 实际上，层和模型拥有两种权重：\n",
    "* “可训练权重”，旨在通过反向传播进行更新，以最大限度地减少模型的损失，例如密集层的内核和偏差。\n",
    "* “不可训练的权重”，意味着在前向传递期间由拥有它们的层更新。 例如，如果您想要一个自定义层保留一个计数器，记录到目前为止已处理的批次数，该信息将存储在不可训练的权重中，并且在每个批次中，您的层会将计数器加一。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "homeless-browser",
   "metadata": {},
   "source": [
    "在 Keras 的内置层中，唯一具有不可训练权重的层是 BatchNormalization 层，我们将在第 9 章介绍。 BatchNormalization 层需要不可训练的权重，以跟踪有关均值和标准差的信息 通过它的数据，以便执行特征归一化的在线近似（您在第 6 章中学到的概念）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "corresponding-swiss",
   "metadata": {},
   "source": [
    "考虑到这两个细节，监督学习训练步骤最终看起来像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "documentary-memory",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_step(inputs, targets):\n",
    "    with tf.GradientTape() as tape:\n",
    "        predictions = model(inputs, training=True)\n",
    "        loss = loss_fn(targets, predictions)\n",
    "    gradients = tape.gradients(loss, model.trainable_weights)\n",
    "    optimizer.apply_gradients(zip(model.trainable_weights, gradients))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "technical-fellowship",
   "metadata": {},
   "source": [
    "### 指标的低级使用"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "controlling-shannon",
   "metadata": {},
   "source": [
    "在低级训练循环中，您可能希望利用 Keras 指标（无论是内置指标中的自定义指标）。 您已经了解了指标 API：只需为每批目标和预测调用 update_state(y_true, y_pred)，然后使用 result() 查询当前指标值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "massive-applicant",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow import keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "featured-european",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result: 1.00\n"
     ]
    }
   ],
   "source": [
    "metric = keras.metrics.SparseCategoricalAccuracy()\n",
    "targets = [0, 1, 2]\n",
    "predictions = [[1, 0, 0], [0, 1, 0], [0, 0, 1]]\n",
    "metric.update_state(targets, predictions)\n",
    "current_result = metric.result()\n",
    "print(f\"result: {current_result:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "subsequent-dietary",
   "metadata": {},
   "source": [
    "您可能还需要跟踪标量值的平均值，例如模型的损失。 您可以通过 keras.metrics.Mean 指标执行此操作："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "polar-geneva",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean of values: 2.00\n"
     ]
    }
   ],
   "source": [
    "values = [0, 1, 2, 3, 4]\n",
    "mean_tracker = keras.metrics.Mean()\n",
    "for value in values:\n",
    "    mean_tracker.update_state(value)\n",
    "print(f\"Mean of values: {mean_tracker.result():.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "growing-spoke",
   "metadata": {},
   "source": [
    "当您想重置当前结果时（在训练时期开始或评估开始时），请记住使用 `metric.reset_state()`。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "emerging-royal",
   "metadata": {},
   "source": [
    "### 完整的训练和评估循环"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "korean-belle",
   "metadata": {},
   "source": [
    "让我们将前向传递、后向传递和指标跟踪组合成一个类似 fit() 的训练步骤函数，该函数接受一批数据和目标，并返回将要显示的日志\n",
    "通过 fit() 进度条："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "handled-reader",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 218 ms (started: 2021-07-28 20:41:16 +08:00)\n"
     ]
    }
   ],
   "source": [
    "model = get_mnist_model()\n",
    "\n",
    "# 准备损失函数。\n",
    "loss_fn = keras.losses.SparseCategoricalCrossentropy()\n",
    "# 准备优化器。\n",
    "optimizer = keras.optimizers.RMSprop()\n",
    "# 准备要监控的指标列表。\n",
    "metrics = [keras.metrics.SparseCategoricalAccuracy()]\n",
    "#     准备一个指标跟踪器来跟踪损失平均值。\n",
    "loss_tracking_metric = keras.metrics.Mean()\n",
    "\n",
    "def train_step(inputs, targets):\n",
    "            \n",
    "#     运行前向传递。 请注意，我们传递了 training=True。\n",
    "    with tf.GradientTape() as tape:\n",
    "        predictions = model(inputs, training=True) \n",
    "        loss = loss_fn(targets, predictions)\n",
    "        \n",
    "# 运行反向传播。 请注意，我们使用了 model.trainable_weights。   \n",
    "    gradients = tape.gradient(loss, model.trainable_weights)\n",
    "    optimizer.apply_gradients(zip(gradients, model.trainable_weights)) \n",
    "\n",
    "    # 跟踪指标\n",
    "    logs = {}\n",
    "    for metric in metrics:\n",
    "        metric.update_state(targets, predictions) \n",
    "        logs[metric.name] = metric.result()\n",
    "        \n",
    "#     跟踪损失平均值。\n",
    "    loss_tracking_metric.update_state(loss)\n",
    "    \n",
    "#     返回指标和损失的当前值。\n",
    "    logs[\"loss\"] = loss_tracking_metric.result() \n",
    "    return logs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "harmful-actor",
   "metadata": {},
   "source": [
    "我们需要在每个时期开始时和运行评估之前重置指标的状态。 这是一个实用函数来做到这一点："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "greek-consumption",
   "metadata": {},
   "source": [
    "> 编写分步训练循环：重置指标"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "unlike-malaysia",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.29 ms (started: 2021-07-28 20:43:02 +08:00)\n"
     ]
    }
   ],
   "source": [
    "def reset_metrics():\n",
    "    for metric in metrics:\n",
    "        metric.reset_states()\n",
    "    loss_tracking_metric.reset_states()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "piano-affairs",
   "metadata": {},
   "source": [
    "我们现在可以布置我们完整的训练循环。 请注意，我们使用 tf.data.Dataset 对象将 NumPy 数据转换为迭代器，该迭代器以大小为 32 的批次迭代数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suburban-importance",
   "metadata": {},
   "source": [
    "> 编写分步训练循环：循环本身"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "sized-acquisition",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Results at the end of epoch 0\n",
      "...sparse_categorical_accuracy: 0.9155\n",
      "...loss: 0.2855\n",
      "Results at the end of epoch 1\n",
      "...sparse_categorical_accuracy: 0.9537\n",
      "...loss: 0.1658\n",
      "Results at the end of epoch 2\n",
      "...sparse_categorical_accuracy: 0.9619\n",
      "...loss: 0.1398\n",
      "time: 4min 20s (started: 2021-07-28 20:48:45 +08:00)\n"
     ]
    }
   ],
   "source": [
    "training_dataset = tf.data.Dataset.from_tensor_slices((train_images, train_labels))\n",
    "training_dataset = training_dataset.batch(32)\n",
    "\n",
    "epochs = 3\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    reset_metrics()\n",
    "    for inputs_batch, targets_batch in training_dataset:\n",
    "        logs = train_step(inputs_batch, targets_batch)\n",
    "    print(f\"Results at the end of epoch {epoch}\")\n",
    "    for key, value in logs.items():\n",
    "        print(f\"...{key}: {value:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sound-variety",
   "metadata": {},
   "source": [
    "这是评估循环：一个简单的 for 循环，它重复调用 test_step 函数，该函数处理单批数据。 test_step 函数只是 train_step 逻辑的一个子集。 它省略了处理更新模型权重的代码，也就是说，所有涉及 GradientTape 和优化器的代码。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "hairy-annex",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation results:\n",
      "...val_sparse_categorical_accuracy: 0.9684\n",
      "...val_loss: 0.1284\n",
      "time: 11.8 s (started: 2021-07-28 21:12:55 +08:00)\n"
     ]
    }
   ],
   "source": [
    "def test_step(inputs, targets):\n",
    "#    请注意，我们传递了 training=False\n",
    "    predictions = model(inputs, training=False) \n",
    "    loss = loss_fn(targets, predictions)\n",
    "    \n",
    "    logs = {}\n",
    "    for metric in metrics:\n",
    "        metric.update_state(targets, predictions)\n",
    "        logs[\"val_\" + metric.name] = metric.result()\n",
    "\n",
    "    loss_tracking_metric.update_state(loss)\n",
    "    logs[\"val_loss\"] = loss_tracking_metric.result()\n",
    "    return logs\n",
    "\n",
    "val_dataset = tf.data.Dataset.from_tensor_slices((val_images, val_labels))\n",
    "val_dataset = val_dataset.batch(32)\n",
    "reset_metrics()\n",
    "\n",
    "for inputs_batch, targets_batch in val_dataset:\n",
    "    logs = test_step(inputs_batch, targets_batch)\n",
    "    \n",
    "print(\"Evaluation results:\")\n",
    "for key, value in logs.items():\n",
    "    print(f\"...{key}: {value:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "approximate-pharmacy",
   "metadata": {},
   "source": [
    "恭喜——你刚刚重新实现了 fit() 和 evaluate()！ 或者几乎：fit() &evaluate() 支持更多功能，包括大规模分布式计算，这需要更多的工作。 它还包括几个关键的性能优化。\n",
    "\n",
    "我们来看一个关键的优化：TensorFlow 函数编译。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "velvet-award",
   "metadata": {},
   "source": [
    "### 使用 tf.function 使其快速"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "actual-detection",
   "metadata": {},
   "source": [
    "您可能已经注意到，尽管实现了基本相同的逻辑，但您的自定义循环的运行速度明显慢于内置的 fit() 和 evaluate() 。 这是因为默认情况下，TensorFlow 代码是逐行执行的，“急切地”执行，很像 NumPy 代码或常规 Python 代码。 Eager Execution 使调试代码更容易，但从性能的角度来看，它远非最佳。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "universal-laugh",
   "metadata": {},
   "source": [
    "将 TensorFlow 代码转换为编译计算图的性能更高，可以以逐行解释代码的方式进行全局优化。 执行此操作的语法非常简单：只需在执行前向要编译的任何函数添加 @tf.function 即可。 像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "descending-credit",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation results:\n",
      "...val_sparse_categorical_accuracy: 0.9684\n",
      "...val_loss: 0.1284\n",
      "time: 2.51 s (started: 2021-07-28 21:19:05 +08:00)\n"
     ]
    }
   ],
   "source": [
    "@tf.function #唯一改变的行\n",
    "def test_step(inputs, targets):\n",
    "#    请注意，我们传递了 training=False\n",
    "    predictions = model(inputs, training=False) \n",
    "    loss = loss_fn(targets, predictions)\n",
    "    \n",
    "    logs = {}\n",
    "    for metric in metrics:\n",
    "        metric.update_state(targets, predictions)\n",
    "        logs[\"val_\" + metric.name] = metric.result()\n",
    "\n",
    "    loss_tracking_metric.update_state(loss)\n",
    "    logs[\"val_loss\"] = loss_tracking_metric.result()\n",
    "    return logs\n",
    "\n",
    "val_dataset = tf.data.Dataset.from_tensor_slices((val_images, val_labels))\n",
    "val_dataset = val_dataset.batch(32)\n",
    "reset_metrics()\n",
    "\n",
    "for inputs_batch, targets_batch in val_dataset:\n",
    "    logs = test_step(inputs_batch, targets_batch)\n",
    "    \n",
    "print(\"Evaluation results:\")\n",
    "for key, value in logs.items():\n",
    "    print(f\"...{key}: {value:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "concerned-reservation",
   "metadata": {},
   "source": [
    "在 Colab CPU 上，我们从运行评估循环所需的 1.80 秒变为仅 0.8 秒。 快多了！"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cellular-harbor",
   "metadata": {},
   "source": [
    "请记住：在调试代码时，最好不要任何 @tf.function 装饰器，而是急切地运行它。 以这种方式跟踪错误会更容易。 一旦您的代码可以运行并且您想使其更快，请将 @tf.function 装饰器添加到您的训练步骤和评估步骤 - 或任何其他对性能至关重要的函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "forced-analyst",
   "metadata": {},
   "source": [
    "### 利用 fit() 与自定义训练循环"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "selective-survivor",
   "metadata": {},
   "source": [
    "在上面的部分中，我们完全从头开始编写自己的训练循环。 这样做为您提供了最大的灵活性，但您最终会编写大量代码，同时错过了许多方便的 fit() 功能，例如回调或对分布式训练的内置支持。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "potential-projector",
   "metadata": {},
   "source": [
    "如果您需要自定义训练算法，但仍想利用内置 Keras 训练逻辑的强大功能，该怎么办？ fit() 和从头开始编写的训练循环之间实际上有一个中间立场：您可以提供自定义训练步骤函数，让框架完成其余的工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "handmade-father",
   "metadata": {},
   "source": [
    "您可以通过覆盖 Model 类的 train_step() 方法来做到这一点。 这是 fit() 为每批数据调用的函数。 然后你就可以像往常一样调用 fit() 了——它会在后台运行你自己的学习算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "competent-aviation",
   "metadata": {},
   "source": [
    "这是一个简单的例子：\n",
    "* 我们创建了一个新类，它是 keras.Model 的子类。\n",
    "* 我们重写方法 train_step(self, data)。 它的内容与我们在上一节中使用的内容几乎相同。\n",
    "* 我们返回一个字典，将度量名称（包括损失）映射到它们的当前值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "fleet-bolivia",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 13.7 ms (started: 2021-07-28 21:32:19 +08:00)\n"
     ]
    }
   ],
   "source": [
    "loss_fn = keras.losses.SparseCategoricalCrossentropy()\n",
    "# 该度量对象将用于跟踪训练和评估期间每批损失的平均值。\n",
    "loss_tracker = keras.metrics.Mean(name=\"loss\")\n",
    "\n",
    "class CustomModel(keras.Model):\n",
    "#     我们覆盖了 train_step 方法。\n",
    "    def train_step(self, data):\n",
    "        inputs, targets = data\n",
    "        with tf.GradientTape() as tape:\n",
    "#             请注意，我们使用 self(inputs, training=True) 而\n",
    "#             不是 model(inputs, training=True)，因为我们的模型是类本身。\n",
    "            predictions = self(inputs, training=True)\n",
    "            loss = loss_fn(targets, predictions)\n",
    "        \n",
    "        gradients = tape.gradient(loss, model.trainable_weights)\n",
    "        optimizer.apply_gradients(zip(gradients, model.trainable_weights))\n",
    "        \n",
    "#         我们更新了跟踪损失平均值的损失跟踪指标。\n",
    "        loss_tracker.update_state(loss)\n",
    "#         我们通过查询损失跟踪器指标返回到目前为止的平均损失。\n",
    "        return {\"loss\": loss_tracker.result()}\n",
    "\n",
    "# 在 model.metrics 属性中列出损失跟踪器指标使模型能够在每个时期的开始和对评估（）的调用开始时\n",
    "# 自动对其调用 reset_states()——因此您不必手工这样做。 您希望跨周期重置的任何指标都应在此处列出。\n",
    "    @property\n",
    "    def metrics(self):\n",
    "        return [loss_tracker]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "flush-smith",
   "metadata": {},
   "source": [
    "我们现在可以实例化我们的自定义模型，编译它（我们只通过优化器，因为损失已经在模型之外定义了），然后像往常一样使用 fit() 训练它。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "forbidden-intermediate",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "1563/1563 [==============================] - 15s 8ms/step - loss: 0.2967\n",
      "Epoch 2/3\n",
      "1563/1563 [==============================] - 14s 9ms/step - loss: 0.1661\n",
      "Epoch 3/3\n",
      "1563/1563 [==============================] - 13s 9ms/step - loss: 0.1410\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f8440078070>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 42.5 s (started: 2021-07-28 21:33:05 +08:00)\n"
     ]
    }
   ],
   "source": [
    "inputs = keras.Input(shape=(28 * 28,))\n",
    "features = layers.Dense(512, activation=\"relu\")(inputs)\n",
    "features = layers.Dropout(0.5)(features)\n",
    "outputs = layers.Dense(10, activation=\"softmax\")(features)\n",
    "model = CustomModel(inputs, outputs)\n",
    "\n",
    "model.compile(optimizer=keras.optimizers.RMSprop())\n",
    "model.fit(train_images, train_labels, epochs=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "constant-starter",
   "metadata": {},
   "source": [
    "注意：\n",
    "* 此模式不会阻止您使用 Functional API 构建模型。 无论您是在构建顺序模型、功能 API 模型还是子类模型，您都可以这样做。\n",
    "* 当您覆盖 train_step 时，您不需要使用 @tf.function 装饰器 - 框架会为您完成。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "changed-extension",
   "metadata": {},
   "source": [
    "现在，指标如何，以及如何通过 compile() 配置损失？ 调用 compile() 后，您可以访问：\n",
    "* self.compiled_loss——你传递给compile()的损失函数。\n",
    "* self.compiled_metrics——你传递的指标列表的包装器，它允许你调用 self.compiled_metrics.update_state() 来一次更新你的所有指标。\n",
    "* self.metrics——你传递给 compile() 的实际指标列表。 请注意，它还包括一个跟踪损失的指标，类似于我们之前手动使用 loss_tracking_metric 所做的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "martial-brooks",
   "metadata": {},
   "source": [
    "因此我们可以这样写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "least-score",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 11.2 ms (started: 2021-07-28 21:40:03 +08:00)\n"
     ]
    }
   ],
   "source": [
    "class CustomModel(keras.Model):\n",
    "    def train_step(self, data):\n",
    "        inputs, targets = data\n",
    "        with tf.GradientTape() as tape:\n",
    "            predictions = self(inputs, training=True)\n",
    "            loss = self.compiled_loss(targets, predictions)\n",
    "            \n",
    "        gradients = tape.gradient(loss, model.trainable_weights)\n",
    "        optimizer.apply_gradients(zip(gradients, model.trainable_weights))\n",
    "        \n",
    "        self.compiled_metrics.update_state(targets, predictions) \n",
    "        \n",
    "        return {m.name: m.result() for m in self.metrics}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "close-africa",
   "metadata": {},
   "source": [
    "让我们试试看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "prospective-mexican",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "1563/1563 [==============================] - 15s 9ms/step - loss: 0.2950 - sparse_categorical_accuracy: 0.9127\n",
      "Epoch 2/3\n",
      "1563/1563 [==============================] - 13s 9ms/step - loss: 0.1661 - sparse_categorical_accuracy: 0.9530\n",
      "Epoch 3/3\n",
      "1563/1563 [==============================] - 13s 8ms/step - loss: 0.1379 - sparse_categorical_accuracy: 0.9621\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f8448063f10>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 41.5 s (started: 2021-07-28 21:41:03 +08:00)\n"
     ]
    }
   ],
   "source": [
    "inputs = keras.Input(shape=(28 * 28,))\n",
    "\n",
    "features = layers.Dense(512, activation=\"relu\")(inputs)\n",
    "features = layers.Dropout(0.5)(features)\n",
    "outputs = layers.Dense(10, activation=\"softmax\")(features)\n",
    "model = CustomModel(inputs, outputs)\n",
    "\n",
    "model.compile(optimizer=keras.optimizers.RMSprop(),\n",
    "              loss=keras.losses.SparseCategoricalCrossentropy(),\n",
    "              metrics=[keras.metrics.SparseCategoricalAccuracy()])\n",
    "\n",
    "model.fit(train_images, train_labels, epochs=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "twenty-princeton",
   "metadata": {},
   "source": [
    "那是很多信息，但现在您已经足够了解使用 Keras 做几乎任何事情。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fresh-macedonia",
   "metadata": {},
   "source": [
    "## 章节总结"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "informative-tumor",
   "metadata": {},
   "source": [
    "* Keras 提供了一系列不同的工作流程，基于复杂性的渐进披露原则。 它们都可以顺利地互操作。\n",
    "* 您可以通过 Sequential 类、Functional API 或通过子类化该类来构建模型。 大多数情况下，您将使用 Functional API。\n",
    "* 通过默认的 fit() 和evaluate() 方法训练和评估模型的最简单方法。\n",
    "* Keras 回调提供了一种在调用 fit() 期间监控模型的简单方法，并根据模型的状态自动采取行动。\n",
    "* 您还可以通过覆盖 train_step() 方法完全控制 fit() 所做的事情。\n",
    "* 除了 fit()，您还可以完全从头开始编写自己的训练循环。 这对于实施全新训练算法的研究人员很有用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "arctic-bubble",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:tensorflow]",
   "language": "python",
   "name": "conda-env-tensorflow-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
